Update:
After being unable to install any versions of Appvolumes on Windows Server 2019 I decided to try Windows Server 2016. To my surprise I faced the exact same problems there, working my way backwards in versions. When I got to 2.13.3 it actually worked (2.13 will not install on 2019):
From that version I was able to upgrade to 2.18 and 4.0.1.
Haven't tried upgrading Windows Server from 2016 to 2019.
Lars