As politely suggested by Mr. Garmin, we drove directly to the parking (3 euro for 6 hours)
and then followed the signs.
It turns out that the entrance is just next to the bridge on its left side, and if you understand German you probably cannot miss it. We figured that the gorge is probably much bigger than it appeared at the first glance (from the bridge), and it would take some time to explore it. But it started raining a little bit, and as it was already a lunch time, we decided to grab a bite before we actually go for a hike.
The Kaiserhaus restaurant was in just 3 minutes walk uphill (it's not the Kaiserhaus, it's just a restaurant with the same name. Kaiserhaus is there too, another few hundred meters uphill, but we did not go there because, again, we are not big fans of museums). In the past year, back home in Canada, we were dreaming (well, not dreaming but definitely talking too much) about having a classic schnitzel and/or Vienna sausages (we are familiar with the concept of both from our previous life), and this is where we actually made this dream a reality first time in this trip:
I ordered the schnitzel, and my wife went for those sausages; both were, well, I would not say amazing, but very very good. What is actually amazing in both Bavaria and Tyrol is wheat beer, and we had that too. You must try it. It does not make sense to go to Bavaria or Tyrol and not to try wheat beer.
In the meantime the raining stopped and we finally went for that hike, where, by the way, we met again that happy elderly couple on their way back. This encouraged us to go till the end of the trail because we knew that people return from it in one piece.
Well, I must say: the gorge is fantastic. Just don't miss that entrance as we almost did. The round trip hike takes about an hour or maybe a bit more. We were wondering what is at the end of it, and there is your spoiler alert: it's just another entrance looking exactly like the one by the bridge, but going in the opposite direction.
If interested, here you can find more pictures and some videos of Kaiserklamm (Emperor’s Gorge).