ffmpeg vault
============

Essentials
----------

### What?

[ffmpeg](https://ffmpeg.org/) is a swiss army knife for everything audio/video. It can do practically every task under the sun, and in fact powers most major dedicated "video players" (VLC, MPC-HC, built-in players in Chrome and Firefox...)[^1]

### How?

If you're on Windows, it's technically possible to install `ffmpeg` and use it directly [^2], but since the windows Command Prompt sucks ass comfort-wise and scripting-wise, it's recommended to just [install Ubuntu as part of the Windows Subsystem for Linux](https://docs.microsoft.com/en-us/windows/wsl/install-win10), and then `apt-get install ffmpeg`.

If you're on Linux, you already know what to do 😎

Techniques
----------

(All commands are expected to be ran in `bash` or a similar Linux shell.)

### Basic conversions

`ffmpeg` is pretty clever, it can correctly guess the codecs and reasonable default settings by the file extension, so all of the following will work as expected (and retain metadata[^3]!):

``` {.bash}
ffmpeg -i video.avi video.mp4
ffmpeg -i video.mp4 video_sound_only.wav
ffmpeg -i video_sound_only.wav video_sound_only.mp3
ffmpeg -i song.flac song.mp3
```

#### mp3 bitrates

"Reasonable" might not be what you want though, especially in the case of mp3, where the default bitrate is V4 (!), i.e. 140-185 kbps.

If you want, for example, V[0], use the `-q:a`[^4] option, like so:

``` {.bash}
ffmpeg -i song.flac -q:a 0 song.mp3
```

More info at: <https://trac.ffmpeg.org/wiki/Encode/MP3>

#### video codecs

Since container/format ≠ codec, you might want to select the codec manually.

While it can reasonably assumed that `mp4` ≅ `h264`, `avi` is a bit more complex. You can list all the supported codecs with `ffmpeg -codecs`[^5], but since there's several hundreds, you better have an idea of what you want to do in the first place.

For example, if you want an `.avi` with xvid codec, you just do:

``` {.bash}
ffmpeg -i original.mp4 -c:v libxvid output.avi
```

#### General codec options

This StackOverflow post explains everything: <https://stackoverflow.com/a/20587693/3833159>

### video // image files // frames

#### images -&gt; video

##### 1. creating a list of images

`ffmpeg` needs a list of images in a text file in a [specific format](https://trac.ffmpeg.org/wiki/Concatenate#demuxer) in order to convert them to a video. There's a couple ways to do this:

``` {.bash}
ls *.jpg | xargs -I xyz echo "file 'xyz'" > list.txt
```

``` {.bash}
for f in *.jpg; do echo "file '$f'" >> list.txt; done
```

It's up to preference, all end up with a list of all JPGs in current directory, in `list.txt`.

##### 2. list to video

``` {.bash}
ffmpeg -f concat -r 30 -i list.txt out.mp4
```

`-f concat` tells `ffmpeg` to handle `list.txt` as a list.

`-r 30` specifies resulting FPS (30 FPS)

`out.mp4` is output file - autodetected as h264-encoded. (`out.avi`, `out.gif`, etc. also work - refer to ffmpeg manual)

#### video -&gt; images

``` {.bash}
ffmpeg -i FILE image%05d.png
```

Where `FILE` is the video file, and `image%05d.png` is the format string for image filenames; this will create `image00001.png`, `image00002.png`, `image00123.png`, etc. (`%05d` means pad with `5` zeroes; `%010d` for padding with `10` zeroes...)

### Streams

`ffmpeg` can also smoothly handle streams, so basic stream capture is pretty trivial, provided you grabbed the playlist/HLS url from somewhere[^6]:

``` {.bash}
ffmpeg -i "https://example.com/playlist.m3u8" my_stream.mp4
```

#### Taking a screenshot of a stream

`-vframes 1` is the option that tells `ffmpeg` to just capture one (i.e. the first) frame of the video - in the case of streams, this means the latest one anyway.

``` {.bash}
ffmpeg -i "https://example.com/playlist.m3u8" -vframes 1 capture.jpg
```

### dropped frame re-interpolation

`ffmpeg` also has a [rich set of filters](https://ffmpeg.org/ffmpeg-filters.html), two of which are of interest for us now:

-   [mpdecimate](https://ffmpeg.org/ffmpeg-filters.html#mpdecimate) - *Drop frames that do not differ greatly from the previous frame in order to reduce frame rate.*
-   [minterpolate](https://ffmpeg.org/ffmpeg-filters.html#minterpolate) - *Convert the video to specified frame rate using motion interpolation.*

The idea is that `mpdecimate` drops all near-duplicate frames, and `minterpolate` re-calculates them using non-duplicate frames that were left.

`mpdecimate`'s defaults are pretty okay, but the result may not look too good if the frame drops are frequent and long. I've had pretty good results using its `max` parameter which limits the amount of frames dropped in a single stretch of video, e.g. `-vf mpdecimate=max=15` which drops at most 15 frames (i.e. half a second assuming 30 FPS), meaning interpolation won't happen everywhere and the video will remain faithfully choppy.

`minterpolate`, on the other hand, defaults to semi-smart motion compensated interpolation, and that *might* just be what you want, but it generally gives pretty funky results. Fortunately, it also has a "blend" mode, which just averages the start and end frames and crossfades them, which gives much more agreeable outputs for simple frame drop situations. It is also generally much faster, I was getting near or above real-time speeds using "blend", whereas motion compensation dropped the processing speed to 0.01x.

**TL;DR**: Full command(s) including the filter pipeline:

``` {.bash}
# Fill out all frame drop gaps
ffmpeg -i choppy_video.mp4 -vf mpdecimate,minterpolate=mi_mode=blend smoother_video.mp4

# Fill out all frame drop gaps no longer than 10 frames:
ffmpeg -i choppy_video.mp4 -vf mpdecimate=max=10,minterpolate=mi_mode=blend smoother_video.mp4

# Motion interpolate the gaps and replicate a bad ketamine trip
ffmpeg -i choppy_video.mp4 -vf mpdecimate,minterpolate smoother_video.mp4
```

### mp4 compatibility

h264 also has "profiles", basically [sets of features](https://en.wikipedia.org/wiki/Advanced_Video_Coding#Profiles) - and it turns out this can make the difference between a file working and not working on some crappy embedded media players, like TVs or pico projectors.

> > The `-profile:v` option limits the output to a specific H.264
> > profile. Some devices (mostly very old or obsolete) only support the
> > more limited Constrained Baseline or Main profiles. You can set
> > these profiles with `-profile:v baseline` or `-profile:v main`.
>
> <https://trac.ffmpeg.org/wiki/Encode/H.264#Profile>

``` {.bash}
ffmpeg -i original.mp4 -profile:v baseline output.mp4
```

And apparently, some players are also sensitive to the pixel format[^7], i.e. can't handle anything else than YUV w/ 4:2:0 chroma subsampling, to fix this use the `-pix_fmt` option as follows:

``` {.bash}
ffmpeg -i original.mp4 -pix_fmt yuv420p output.mp4
# or, with the profile settings...
ffmpeg -i original.mp4 -profile:v baseline -pix_fmt yuv420p output.mp4
```

No silver bullet, you'll just have to try different things for different devices. A database of crappy players and appropriate `ffmpeg` settings would be great.

### random

<https://ottverse.com/ffmpeg-drawtext-filter-dynamic-overlays-timecode-scrolling-text-credits/>

---

# Footnotes

[^1]: See https://en.wikipedia.org/wiki/FFmpeg#Projects_using_FFmpeg

[^2]: from <https://ffmpeg.zeranoe.com/builds/>

[^3]: like ID3 tags and their FLAC, OGG, WAV, etc. equivalents

[^4]: read as -quality:audio

[^5]: and respectively, formats with `ffmpeg -formats`

[^6]: like <https://addons.mozilla.org/en-US/firefox/addon/hls-stream-detector/>; or just press F12, check the Network tab, and look carefully

[^7]: From: https://trac.ffmpeg.org/wiki/Encode/H.264#Encodingfordumbplayers