Register | Log-in

> Help

Automatic text to speech subtitles and translation

1
Automatic text to speech subtitles and translation
Mushroom
2023-01-23 20:37:53
For those that are interested, Subtitle Edit has just added some new features.
Among these is a "text to speech" recognition capability.  This was recently added, and I am probably going to be making a lot more subtitles in the future because of it.
If anybody wants to play around with this feature, the program is free and can be downloaded here:
https://github.com/SubtitleEdit/subtitleedit
You will also need the models that are required to do the text to speech.  Those are also free, and can be found here:
https://alphacephei.com/vosk/models
Simply grab the ones you want.  But remember, if you get a foreign language one, it will not convert those to English.  Don't worry, we will do that later.
In the Subtitle Edit directory, make a Vosk directory if it does not already exist, then extract the files/directories you just got and put them in there.  At this time, I have the vosk-model-en-ua-0.22 and the vost-model-ja-0.22 directories there as those are the languages I am most interested in converting.
Now if you want to try the capture feature, the best way I found is to make a blank text file with the name of the video file, but ending in SRT.  You then open it with Subtitle Edit, and ignore the warning it is a short file.  Then once the program is open, go to the VIDEO tab and open the video that you want to convert.  Once it is open, return to the video tab again and select "Audio to Text (Vosk/Kaldi)".  A box is going to appear telling you to pick a model, simply use the pull-down tab to select the one that you want to use, and select "Generate".
Now it will take some time, I find about 30 minutes per hour of video on average.  When it is done, you will have a fairly crude text to speech of the audio of the movie.  I would not consider it good enough to share, but if you want it for your own use or as a start for one that you want to clean up, it is a great start.
Now if it was in another language, don't worry.  At the top is the "Auto-Translate" tab.  Just select that, put in what language you want to translate it to, and in a few moments it will be done.
Either way, at this point save the results, and you now have a subtitle ready to use.  Now for the translation, the program does use Google Translate.  But feel free to use another if you wish.  And there is a limit to free translation, I want to say it is 4 or 5 an hour.  I have hit it a few times, but as most times I do this I am doing a conversion then hours worth of editing I have hit that in a long time.  Just wait an hour or so then try it again.  Or if you already subscribe to the translation service, put in your credentials and use it as much as you like.

1