jump to navigation

Scripture Speech Search April 13, 2016

Posted by P, J, or J in Uncategorized.
trackback

–six stages towards the not yet possible

audiosearchedscripture1

Peter Brassington , April 12, 2016

Speech recognition already exists for several major languages converting audio to text and then searching for the text. I was not aware that anyone has yet built this into a Bible app but within 1-5 years I expect 60-80% of the planet to be able to search their text or audio Bibles just by asking “OK Bible where does it say…” or “what does the Bible say about…”.

Speech recognition for minority languages may take another couple of years but I fully expect audio materials to be searchable by voice, so that languages may be fully searchable in digital media without needing to be written.

A new addition to Scripture App Builder allows for rapid automated comparison of text to recorded speech so that text can be highlighted along with an audio Bible.

My simplistic understanding is:

  • Text is converted to machine generated audio (not good quality text to speech but good enough for the next task)
  • The machine generated audio is compared to recorded audio Bible
  • Original text can then be matched phrase by phrase to the audio Bible

So my question was. Can a similar process be used to allow the users speech to be compared to the audio Bible and used to find passages or references?

And can data used in the text to speech and audio comparison be used to create ‘profiles’ with which to easily do the same with other text and audio books, allowing further applications to be developed and increasing the amount of data for a language.

Perhaps that’s not quite so easy yet, but here are some other things that are happening now.

 

Which of these are already happening or are currently possible but not yet widely implemented

Stage 1: Phone app learns to distinguish up to six separate commands

Eg prerecorded audio says (in target language). “Please say RUN”, records and compares user saying “RUN” and is able to distinguish that from other recorded/compared commands. Picking words in the target language that sound sufficiently different to each other this was possible to a degree with speech recognition software in the 1980’s running on machines with less than 64Kb of memory.

Uses:

  • Simple games eg commanding a donkey to walk along a track and do simple tasks (slower reaction needed than for a racing game).
  • Navigation through a menu to locate a passage in the Bible. Similar to working through a phone menu
  • Responding to multi-choice audio quiz

 

Stage 2: User repeats phrases and phone extracts keywords. Audio samples are reduced to simple digital form for fuzzy matching. Rather than a match for the whole phrase app builds up a library of 20-100 words or distinguishable phonemes. Comparison may be based on probability

Uses: search of a collection of stories/scenes.

Eg after a scene the audio prompt says “if you want to find this scene again say ,‘JESUS CALMS THE STORM’ please say now ‘JESUS CALMS THE STORM’

Assuming you used some of the small selection of words to be compared perhaps the app decides there is an 85% probability you said “Jesus”, 40% probability you said “calms”, 58% probability you said “storm”. Given the combination from the 50 scenes/stories  available in our collection you want the one labeled “Jesus Calms the Storm”

Stage 3: Collect and crowdsource data from apps 1&2. Is their sufficient similarity not to need to ask user to supply their own initial recording, unless it needs to clarify, “did you say…”
Uses: stronger sense that phone ‘understands’ a few words of you language. Ground laid for gathering more data for larger recognizable vocabulary.

Stage 4: Crowdsourced larger library of words and phrases through activities that require users to repeat phrases.

Uses: Audio Quizzes / Story Memorization eg “What happened to the blind man? Either say ‘JESUS TOLD HIM HE WAS CURSED’ or ‘JESUS HEALED THE BLIND MAN’”

If include geographic data does variation help to map different dialects and regional variation?

Stage 5: Significant database of keywords allow search of larger content set, eg seach through 50-500 scenes based on probability of key words (without the need for the user to record them after each one as in stage 2)

Stage 6:  With future advances in Google voice technology over the next five years how many searches from how many speakers will it take for it be possible to search for audio content in the language from the entire audio Bible, or even on the public internet?

mcommd

30 years ago 16 and 48K computers could be taught to understand speech (badly)

Some of this may not be possible yet but how likely did it seem back in the 1980’s that your phone would be a powerful computer or that the first speech recognition software would lead to being able to ask your phone all the things you can ask now.

How many of these ideas are already reality? How long until the next 1000 minority languages can make use of such technology?

Advertisements

Comments»

1. Parke - April 14, 2016

Interesting thoughts. It could essentially be a much more portable and low-bandwidth way to accomplish something like this. http://blog.youversion.com/2015/11/the-bible-app-is-now-available-on-amazon-echo/

This sounds like an idea you should drop here under “feature suggestion” or in a developer’s hands. http://software.sil.org/scriptureappbuilder/support/

Liked by 1 person

2. Who viewed what? | Pete, Jennifer & Joel - November 2, 2016

[…] Each looked at a different page. One was a lighthearted look at whether pets go to heaven (sorry if you’ve just lost a pet), some idioms from around the world, Bible translation statistics from 2015, and thoughts on voice searchable digital Bibles. […]

Like


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: