Access to the internet is by far the most powerful learning tool anyone can have, and Khan Academy videos were probably the first thing that opened my eyes to this truth as a kid. Growing up, I never really enjoyed doing homework but these videos were one of the few things that really got me into “learning”. I don’t know if it was the visuals or the fact that I could pause and rewind at will, but it was far more engaging compared to any other elementary school lesson I was taught.

I was fortunate enough to have English as my first language growing up but it turns out that English isn’t the native language for 71% of internet users. That means even today, most kids won’t be able to experience the same feeling I had. This is why dubbing is such a hot topic in education.

Traditional dubbing can be expensive and time-consuming due to the labor costs of professional translators and voice actors. It’s why most education content out there isn’t dubbed by default. Within the last year, AI-powered dubbing has changed this because it completely automates this process, which means 10-100x lower cost and turnaround times.

In a previous post, we spoke on the 1.0 launch of Sieve’s Dubbing API but having worked with a ton of education customers lately, I was inspired to write a quick post about how seamless dubbing with Sieve could be, especially in an environment of tight budgets and growing content libraries. And to prove the point, I decided to dub a whole Khan Academy course!

Bulk-downloading Khan Academy videos

Disclaimer: Bulk-downloading videos for most content libraries is against TOS. This is for demo purposes only. Please support Khan Academy by donating here.

khan-dl is an open-source fork of popular YouTube downloading tools which can be used to bulk-download Khan Academy course content.

Install khan-dl

You can install khan-dl with the following command.

pip install khan-dl -U

Download a course

You can then download a specific course by simply passing in the link after -c. This command might take a while depending on the size of the course.

$  khan-dl -c "https://www.khanacademy.org/math/early-math"
 _  __ _   _     _     _   _         ____   _
| |/ /| | | |   / \   | \ | |       |  _ \ | |
| ' / | |_| |  / _ \  |  \| | _____ | | | || |
| . \ |  _  | / ___ \ | |\  ||_____|| |_| || |___
|_|\_\|_| |_|/_/   \_\|_| \_|       |____/ |_____|

Looking up https://www.khanacademy.org/math/early-math...
Course URL: https://www.khanacademy.org/math/early-math

Generating Path Slugs...

Collecting Youtube IDs: 100.0% [========================================================================================================================================>]   4/  4 eta [00:00]
Downloading Videos:   0.0% [>                                                                                                                                          ]   0/ 75 eta [?:??:??]

Automated dubbing with the Sieve API

You should now have a folder with all the course content available. If you just copied the command above, your folder should be something like Early_math_review. Now we can use Sieve to dub it.

Install Sieve Python Package

pip install sievedata

Create a Sieve Account

You can create an account here.

Log in to Sieve in your Terminal

Find your Sieve API key that is available here and paste it into the following command.

sieve login

Run the Script

Now save this script on your laptop and change line 19 to point to the directory containing all your videos. This script finds all the videos in the directory, dubs them into Spanish, and then saves them in a new folder with _dubbed as a new suffix.

import sieve
import os
import concurrent.futures
from pathlib import Path

ROOT_DIR = "/path/to/folder/of/videos"
MAX_VIDEOS = 5 # Set to 0 to dub all videos, otherwise set to the number of videos to dub

def dub_video(
    video_path
):
    source_file = sieve.File(path=video_path)

    dubbing = sieve.function.get("sieve/dubbing:fe53e46f")
    output = dubbing.run(source_file, target_language="spanish")
    video_output = list(output)[0]
    return video_output.path

def process_video(video_path):
    output_path = video_path.replace(ROOT_DIR, ROOT_DIR + '_dubbed')
    os.makedirs(os.path.dirname(output_path), exist_ok=True)
    dubbed_path = dub_video(video_path)
    os.rename(dubbed_path, output_path)

def dub_all_videos(root_dir, max_workers=20):
    video_paths = []
    for root, _, files in os.walk(root_dir):
        for file in files:
            video_paths.append(os.path.join(root, file))

    if MAX_VIDEOS > 0 and MAX_VIDEOS < len(video_paths):
        video_paths = video_paths[:MAX_VIDEOS]

    with concurrent.futures.ThreadPoolExecutor(max_workers=max_workers) as executor:
        futures = [executor.submit(process_video, video_path) for video_path in video_paths]
        concurrent.futures.wait(futures)

dub_all_videos(ROOT_DIR)

And Voilà! Sieve’s infrastructure handles the parallel requests with ease and all the dubbed content should be available locally within a couple minutes (depending on how large the course is).

Here’s a video from the high school biology course I dubbed into Spanish.

Sieve Dubbing for Education

Since launch, our Dubbing API has gained adoption by many education / LMS platforms. While there’s many options available (manual and automated), Sieve’s solution stood out for a few key reasons.

• Quality: Translations can be of varying lengths which can cause audio to be out-of-sync with the original video and text-to-speech models can hallucinate quite a bit. This causes many automated dubbing solutions to be poor options when considering the quality you come to expect from educational content. Sieve’s pipeline gets past these issues better than any other automated solution on the market.
• Flexibility: Unlike most dubbing solutions out there, Sieve’s core focus is in selling to developers who want to integrate various capabilities into their end product experience. This means many options of voice and translation engines to make tradeoffs on cost, quality, and speed along with a seamless developer experience.
• Personalization: Language can be complex and must be catered well to specific dialects, styles, and tonality in which people speak. Sieve dubbing allows users to specify styles as prefixes the target languages (i.e. brazilian portuguese or gen z english) which allows for fine-grained personalization in how videos are translated.
• Translation Corrections: Regardless of how far AI models have gotten, they’re still not perfect. Sieve allows users to override translations of specify the way certain words or phrases should get translated using the translation_dictionary.

Give us your feedback

If this is interesting to you, check out the app via our playground here, join us in Discord, or reach out to us at contact@sievedata.com. Our team is always looking for feedback and ways we can continue to improve the application, especially if you have use cases related to education.

Also, while we don’t work directly with schools or non-profits, we’d love to get a sense of the tools you use today and how we could integrate with them. We’ll even throw free credits in exchange for this feedback :)