Researcher reviewing a video transcript at a monitor

To summarize a video with a large language model (LLM), turn the video into text first, then give that text to the model. For most videos that means getting the transcript, checking that it fits the model’s context window, and asking for a summary in the format you need. When the transcript is too long, you split it, summarize the pieces, and summarize the summaries. This guide covers each step, the model options, what to do when a video has no captions, and how to keep summaries accurate.

Meetclaras
Get to the key points faster
Claras brings transcripts, AI-generated summaries, clickable chapters, and video chat to YouTube, including videos without captions.
Explore Claras

TL;DR

Table of Contents

How an LLM summarizes a video

An LLM reads text. It does not watch a video the way you do. So the practical question is always how to turn the video into something the model can read: a transcript, a set of captioned frames, or a native video input.

There are three routes:

  1. Transcript route. Take the spoken words and summarize them. This covers lectures, interviews, tutorials, podcasts and talks, where the meaning lives in speech. It is the cheapest and the most common route.
  2. Native multimodal route. Send the video itself to a model that accepts video. This helps when slides, charts, demos or on-screen code matter. It is the most expensive route per video, according to a survey of video understanding with LLMs.
  3. Hybrid route. Combine the transcript with captions for a few sampled frames. Research on this is active: a 2025 IJCNLP paper builds a screenplay-like representation of TV episodes from dialogue and clip captions, and reports summaries with 20% more relevant visual information while using 75% less of the video as input.

Start with the transcript route. Move to the others only when a transcript-only summary misses things you can see on screen.

The transcript workflow, step by step

Here is the standard workflow for a YouTube video or any video with usable text.

Step 1: Get the transcript

Pull the captions, or transcribe the audio if there are none. A tool such as the YouTube Subtitle Downloader gives you the text as SRT or TXT. Keep the timestamps if you can. They let you check any claim in the summary later.

Man listening to video audio for transcription

Step 2: Check the length against the context window

Count the tokens in the transcript and compare them with the model’s context window. One published walkthrough of this approach, a Streamlit script from 2024, leaves headroom: it treats about 195,000 tokens as the limit for a model with a 200,000-token window. As a rough guide, spoken English runs at roughly 150 words per minute, so an hour of talk is around 9,000 words. Many current models handle that in one pass, but cost and quality can still push you to split it.

Step 3: Choose stuff or map-reduce

The two common strategies have simple names:

Strategy How it works Best for
Stuff Put the whole transcript in one prompt Short videos where the transcript fits comfortably
Map-reduce Split into chunks, summarize each, then summarize the summaries Long videos, lectures, livestreams, multi-hour talks

A LangChain guide recommends map-reduce for most pipelines because chunks can be processed in parallel and length stops being a limit. It treats the stuff approach as suitable only for videos under about 20 minutes.

Research backs the chunking idea. A KDD 2024 paper showed that pipelines combining recursive and extractive summarization let LLMs summarize transcripts of more than one hour without a hard token limit. The baseline models it compared against could only handle about 512 tokens.

Step 4: Write a prompt that sets the rules

A good summary prompt has four parts: the role, the source rule, the output format and the length. For example:

Summarize this transcript for a product manager. Use only what is in the transcript. Give a three-sentence overview, then five key points with timestamps, then any numbers or names mentioned. If something is unclear, say so.

The source rule matters most. One local pipeline described in a 2026 write-up prompts the model not to add information absent from the transcript, and uses a low temperature of 0.2 for factual consistency.

Step 5: Run the second pass

In map-reduce, the final step merges the chunk summaries. Ask for a deduplicated list, ordered as in the video, with timestamps kept. Then ask for a version in the format you need: bullets, study notes, a blog outline or a script for a short clip.

What if the video has no captions?

Many videos have no usable captions, and an LLM cannot summarize what it cannot read. The fix is speech-to-text first.

Speech-to-text is not perfect. A 2026 study of Whisper on 1,847 oral-history testimonies found a mean word error rate of 15.28%, with most files rated fair or better. The errors clustered around disfluencies, interrupted speech and spelling conventions in particular languages. For clean, single-speaker audio the picture is usually better, but names, jargon and crosstalk are where you should look first.

If you want this handled for you, Claras transcribes videos even without captions, and our summary and chat features then work from that transcript.

Model options for a YouTube summarizer

Your options fall into three groups.

Hosted frontier models

Paste or upload the transcript into a general chat model, or call one through an API. This is the quickest path and gives strong summaries. Context windows are large, so most single videos fit in one pass.

Models that take video directly

The Gemini API is the clearest example. According to Google’s video understanding documentation, you can pass a public YouTube URL, the feature is in preview, the free tier has an 8-hour-per-day cap, and Gemini 2.5 and later models accept up to 10 videos per request. The same page lists four input methods: the File API, Cloud Storage registration, inline data under 100 MB, and YouTube URLs for public videos. By default the model samples visuals at one frame per second.

Local and open models

For privacy or cost control, run everything on your own hardware. A typical stack uses yt-dlp to download, Whisper through faster-whisper to transcribe, and a local model through Ollama to summarize. The 2026 local pipeline write-up splits audio into 3-minute chunks with 5% overlap so words at the boundaries are not lost. A May 2026 build on a mini-PC reports transcribing a 10-minute video in 25 seconds and summarizing it in 44 to 72 seconds.

Smaller models also work. One guide from February 2026 uses Flan-T5-Base with 1,200-character chunks and a sliding window to turn subtitles into notes.

Option Input Strength Trade-off
Hosted chat model Transcript text Fast, high quality You supply the transcript
Gemini API Public YouTube URL or file Skips the transcript step Public videos only, preview limits
Local Whisper plus local LLM Audio or video file Data stays with you Setup time and hardware
Multimodal pipeline Frames plus transcript Captures slides and visuals Highest compute

Build a custom summarizer: a simple blueprint

If you are an AI builder who wants control over the model, this is the shape most custom summarizers take:

  1. Ingest. Accept a URL, fetch captions, or download audio.
  2. Transcribe. Use captions when they exist, Whisper when they do not.
  3. Chunk. Split by tokens or by chapter, with a little overlap.
  4. Summarize chunks. Run the same prompt on each chunk, in parallel.
  5. Merge. Summarize the summaries, keeping timestamps.
  6. Export. Send the result to notes, a knowledge base or a doc.

Keep the model swappable. Summary quality and price differ across models, and you will want to change them as better ones appear. This is why Claras supports bringing your own AI model, alongside bulk processing for playlists.

Keeping summaries accurate

The main risk is hallucination: the model writes content that the source does not support. The video understanding survey names it as a key challenge, along with difficulty handling long videos and sensitivity to input order when summarizing several videos together.

You can reduce the risk in practice:

Grounding summaries in the source is also how good study habits work. If you are turning summaries into revision material, this guide on summarizing lecture notes shows how to move from condensed text to something you remember.

A note on cost and privacy

Cost scales with what you feed the model. A 2026 survey on VideoLLM efficiency finds that encoders processing hundreds of high-resolution frames can dominate total cost unless frames are subsampled, and that keeping about a quarter of the visual token budget often preserves near-baseline accuracy. Transcript-only summaries sidestep most of that.

Privacy matters when videos are private, internal or confidential. The video understanding survey flags the exposure of private information without consent as a concern. For those files, keep processing local or use a service whose handling you have reviewed. Public YouTube content is a different case from a recorded internal meeting.

Our perspective

Most people think summarizing a video is a prompt problem. In our experience it is a coverage problem first. If the transcript is missing, wrong or buried on another site, no prompt fixes it.

That is why we built Claras around the transcript. It is designed to transcribe and summarize nearly all YouTube videos, including those without captions, and put the result on the YouTube page itself. From there, clickable chapters, summaries and chat with the video all work from the same full transcript, so answers stay tied to what was said.

We also think builders and researchers should not be locked into one model. Bulk processing for playlists, export to common formats and knowledge platforms, and the option to use a custom AI model are part of the design for that reason. The summary is the start of your workflow, not the end.

the author

Skip the pipeline with Claras

If you want the result without building the stack, Claras is a browser tool and Chrome extension that gives you the transcript, an AI summary, clickable chapters, comment summaries and chat with the video, in over 70 languages. Try the free YouTube Video Summarizer first, or see all free YouTube tools.

Meetclaras

There is a limited free plan. Paid plans are Pro at $4 per month ($48 per year) and Unlimited at $10 per month ($120 per year). Details are on the pricing page. For a wider look at the options, read our ranking of ways to summarize a YouTube video.

FAQ

Some models can, and some cannot. Gemini’s API accepts public YouTube URLs directly. Other chat models usually need the transcript pasted in. If a link fails, get the transcript and paste that instead.

How long a video can an LLM summarize?

It depends on the context window. If the transcript fits, one prompt works. If not, use map-reduce: summarize chunks, then summarize the summaries. Research has shown transcripts of more than an hour can be handled this way.

What is the best way to summarize a video with no subtitles?

Transcribe the audio first, with Whisper or a hosted speech-to-text service, then summarize the text. Check names and technical terms in the transcript before you trust the summary. A tool that transcribes for you saves the extra step.

How do I stop an LLM from making things up in a summary?

Tell it to use only the transcript, ask for timestamps with each point, and keep the temperature low. Then click two or three timestamps to confirm the claims that matter. Correcting transcript errors first also helps.

Sources