shaking of video after download from YouTube

Simply watch the following video on YouTube, and after downloading with 4K (in 1080p):
https://www.youtube.com/watch?v=aWFWhlUjapE

The YouTube version has a shake at the start (as the camera gets accidentally hit) but otherwise is steady and nice. The 4K download propagates this initial camera shake through the entire video! I have seen this with several videos, but the problem is particularly prominent here.

The problem is entirely reproducible, e.g., with this particular video. And since the YouTube video is steady (but for the time of the actual shake) it should be possible for 4K to match this.