Qubes OS images - 'distrust the infrastructure' with PGP

GPG Qubes OS

Intro

I’ve been using Qubes OS for almost a year now, and today, I’m setting up a new notebook with it. Qubes OS is an operating system focused on security, so their installation process as well, encourages to distrust all intermediaries and rely solely on cryptographic trust. While I don’t expect there are some intermediaries watching my traffic and just waiting to trick me, I find following the Qubes best practices a good exercise and refresher of how to use GPG correctly when publishing binaries online. At the end, following their guides is learning in practice from some of the best in the security field.

What I’m sharing here then are some notes and printouts documenting and explaining the process of preparing installation USB stick with Qubes OS. How to download images, verify their authenticity and how to burn it. Basically it’s a derivative of information you can find on these two:

Qubes Master Signing Key

Qubes OS team uses different PGP keys to sign their releases. If you go to their downloads page to download image, beside the image, and digest files, you will find a release key. But in the world where we distrust the infrastructure we cannot trust the release key placed there, not the rest of the files. We cannot trust Qubes OS website. How this is solved? So the solution to lack of trust is, is delegation of trust to another key - the Qubes Master Signing Key (QMSK) (0x427F11FD0FAA4B080123F01CDDFA1A3E36879494). This key is published on their site. It is also available on key servers. But why would I trust the file with the key I import? Well, because assuming fingerprint collisions are practically impossible, you can trust the fingerprint. After importing the key, you check the fingerprint, and you can confront it with the fingerprint published in various places.

This is a quote from Qubes OS website:

But how do you know that this is the real fingerprint? After all, this website could be compromised, so the fingerprint you see here may not be genuine. That’s why we strongly suggest obtaining the fingerprint from multiple independent sources in several different ways.

Here are some ideas for how to do that:

OK. Let’s start.

For simplicity, I use a temporary GnuPG home:

% export GNUPGHOME=$(mktemp -d)

% gpg -k
gpg: keybox '/tmp/tmp.3unMXuJY6w/pubring.kbx' created
gpg: /tmp/tmp.3unMXuJY6w/trustdb.gpg: trustdb created

% gpg --version
gpg (GnuPG) 2.4.0
libgcrypt 1.10.2-unknown
Copyright (C) 2021 Free Software Foundation, Inc.
License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: /home/user/.gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

As we see gpg -k doesn’t list any keys. Notice also that despite I use gpg it is gpg in version 2.4.0.

Next I import the Qubes OS Master Signing Key (QMSK). Where to get it from?

Here are some example download locations:

I import it and after confirming the fingerprint is correct, I mark it as trusted.

% gpg --fetch-keys https://keys.qubes-os.org/keys/qubes-master-signing-key.asc
gpg: requesting key from 'https://keys.qubes-os.org/keys/qubes-master-signing-key.asc'
gpg: key DDFA1A3E36879494: public key "Qubes Master Signing Key" imported
gpg: Total number processed: 1
gpg:               imported: 1

% gpg --edit-key 0x427F11FD0FAA4B080123F01CDDFA1A3E36879494
gpg (GnuPG) 2.4.0; Copyright (C) 2021 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.


pub  rsa4096/DDFA1A3E36879494
     created: 2010-04-01  expires: never       usage: SC
     trust: unknown       validity: unknown
[ unknown] (1). Qubes Master Signing Key

gpg> fpr
pub   rsa4096/DDFA1A3E36879494 2010-04-01 Qubes Master Signing Key
 Primary key fingerprint: 427F 11FD 0FAA 4B08 0123  F01C DDFA 1A3E 3687 9494

gpg> trust
pub  rsa4096/DDFA1A3E36879494
     created: 2010-04-01  expires: never       usage: SC
     trust: unknown       validity: unknown
[ unknown] (1). Qubes Master Signing Key

Please decide how far you trust this user to correctly verify other users\' keys
(by looking at passports, checking fingerprints from different sources, etc.)

  1 = I don't know or won't say
  2 = I do NOT trust
  3 = I trust marginally
  4 = I trust fully
  5 = I trust ultimately
  m = back to the main menu

Your decision? 5
Do you really want to set this key to ultimate trust? (y/N) y

pub  rsa4096/DDFA1A3E36879494
     created: 2010-04-01  expires: never       usage: SC
     trust: ultimate      validity: unknown
[ unknown] (1). Qubes Master Signing Key
Please note that the shown key validity is not necessarily correct
unless you restart the program.

gpg> q

% gpg -k "Qubes Master Signing Key"
pub   rsa4096 2010-04-01 [SC]
      427F11FD0FAA4B080123F01CDDFA1A3E36879494
uid           [ultimate] Qubes Master Signing Key

OK. We got the Qubes Master Signing Key imported and verified it is authentic. Now I can download images and verify them.

Download

First I download the files from Qubes OS downloads page or, alternatively from torrent. For simplicity, I download them directly:

% curl -O https://keys.qubes-os.org/keys/qubes-release-4.2-signing-key.asc
% curl -O https://ftp.qubes-os.org/iso/Qubes-R4.2.0-rc4-x86_64.iso.DIGESTS
% curl -O https://ftp.qubes-os.org/iso/Qubes-R4.2.0-rc4-x86_64.iso.asc
% curl -O https://ftp.qubes-os.org/iso/Qubes-R4.2.0-rc4-x86_64.iso

Verify the image with Qubes release key

Among the files downloaded we can find the release key, here it is qubes-release-4.2-signing-key.asc. Let’s import it:

% gpg --import qubes-release-4.2-signing-key.asc
gpg: key E022E58F8E34D89F: public key "Qubes OS Release 4.2 Signing Key" imported
gpg: Total number processed: 1
gpg:               imported: 1
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   1  signed:   1  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: depth: 1  valid:   1  signed:   0  trust: 1-, 0q, 0n, 0m, 0f, 0u

% gpg -k "Qubes OS Release"
pub   rsa4096 2022-10-04 [SC]
      9C884DF3F81064A569A4A9FAE022E58F8E34D89F
uid           [  full  ] Qubes OS Release 4.2 Signing Key

% gpg --check-signatures "Qubes OS Release"
pub   rsa4096 2022-10-04 [SC]
      9C884DF3F81064A569A4A9FAE022E58F8E34D89F
uid           [  full  ] Qubes OS Release 4.2 Signing Key
sig!3        E022E58F8E34D89F 2022-10-04  Qubes OS Release 4.2 Signing Key
sig!         DDFA1A3E36879494 2023-06-03  Qubes Master Signing Key

We see Signing Key is signed by Qubes Master Signing Key - line sig! DDFA1A3E36879494 2023-06-03 Qubes Master Signing Key.

What matters is the line with a sig! prefix showing that the QMSK has signed this key. This verifies the authenticity of the RSK. Note that the ! flag after the sig tag is important because it means that the key signature is valid. A sig- prefix would indicate a bad signature, and sig% would mean that gpg encountered an error while verifying the signature.

We also can see the trust level is full - since we trust QMSK and QMSK signed the release key, the trust is transitive, so we see the release key as trusted.

Now having established trust to the release key provided there, we can verify the image. Qubes OS team provides a image signature (Qubes-R4.2.0-rc4-x86_64.iso.asc). With it we can verify both integrity and authenticity of the image file now:

% gpg -v --verify Qubes-R4.2.0-rc4-x86_64.iso.asc Qubes-R4.2.0-rc4-x86_64.iso
gpg: enabled compatibility flags:
gpg: Signature made Thu Oct 12 22:57:35 2023 -03
gpg:                using RSA key 9C884DF3F81064A569A4A9FAE022E58F8E34D89F
gpg: using pgp trust model
gpg: Good signature from "Qubes OS Release 4.2 Signing Key" [full]
gpg: binary signature, digest algorithm SHA256, key algorithm rsa4096

The image is OK.

Alternative way to confirm integrity and authenticity

In the previous step we verified both: integrity and authenticity in one command. Alternatively, if we like we can verify integrity separately from authenticity.

Among the files we can find a file Qubes-R4.2.0-rc4-x86_64.iso.DIGESTS . It contains four checksums and signature of the content. All in one file.

To verify integrity we can run any of the four commands:

% md5sum -c Qubes-R4.2.0-rc4-x86_64.iso.DIGESTS
Qubes-R4.2.0-rc4-x86_64.iso: OK
md5sum: WARNING: 20 lines are improperly formatted

% sha1sum -c Qubes-R4.2.0-rc4-x86_64.iso.DIGESTS
Qubes-R4.2.0-rc4-x86_64.iso: OK
sha1sum: WARNING: 20 lines are improperly formatted

% sha256sum -c Qubes-R4.2.0-rc4-x86_64.iso.DIGESTS
Qubes-R4.2.0-rc4-x86_64.iso: OK
sha256sum: WARNING: 20 lines are improperly formatted

% sha512sum -c Qubes-R4.2.0-rc4-x86_64.iso.DIGESTS
Qubes-R4.2.0-rc4-x86_64.iso: OK
sha512sum: WARNING: 20 lines are improperly formatted

The warnings here can be ignored. Remember this file contains only one line that each of the hashing function can understand. The rest is noise for it. If you find one OK, it means the file is OK.

Since the same file contains signature, we can check whether it is authentic:

% gpg -v --verify Qubes-R4.2.0-rc4-x86_64.iso.DIGESTS
gpg: enabled compatibility flags:
gpg: armor header: Hash: SHA256
gpg: original file name=''
gpg: Signature made Thu Oct 12 22:59:05 2023 -03
gpg:                using RSA key 9C884DF3F81064A569A4A9FAE022E58F8E34D89F
gpg: using pgp trust model
gpg: Good signature from "Qubes OS Release 4.2 Signing Key" [full]
gpg: textmode signature, digest algorithm SHA256, key algorithm rsa4096

Looks good.

This way we first verified integrity with hashing functions, and then authenticity.

Prepare USB stick

You don’t download the Qubes OS image just to verify it. I put therefore this for completeness of preparing a installation USB stick:

sudo dd if=Qubes-R4.2.0-rc4-x86_64.iso of=/dev/diskX status=progress bs=1M conv=fsync

You need to replace /dev/diskX with the right device. The command lsblk can help you with figuring out the device name at system.

Summary

The idea to distrust infrastructure to me sounds like a healthy practice if you care about security. Qubes OS is a great example of how you can distrust everything including SSL CA, and still be sure what you download comes from the author. The key here is the master key. Establish it, use it, share it.

Mine is: F347 6719 D066 1632 D61D 0B3B 6762 037B 92E2 622E.