Asymmetric Encryption#

ToDo:

Add illustration for Asymmetric Encryption - Similar to this and this.
Add illustration for digital signature - Similar to this and this and this
Explain the SSL Handshake - Similar to this
Add appendix using requests to show self-signed warnings.
Add relevant resources at the end.

Asymmetric encryption does not have passwords or keys like symmetric but rather it splits the security into a pair of keys, one used to encrypt (usually called Public Key) and another used to decrypt (usually call Private Key). As the name implies one can be shared publicly and the other should be kept secret. This method is also known as Public-key cryptography.

The keys are sequences of bytes generated together and are mathematically linked, they are also called “Key Pair”. Each party in the communication should have its own key pair and have their Public Keys shared. This means that there is no need for a “secure channel” to exchange keys as in symmetric encryption since the Public key are by design sharable.

This type of encryption is used nowadays in many applications, ranging from the commonplace SSH protocol to the trendy Bitcoin transactions.

A party can generate as many Key Pairs as needed, meaning that, in case of a Private Key being compromised, a new Key Pair can be generated.

Public Key Encryption != Digital Signature#

The idea of a Digital Signature is basically reversing Public Key Encryption, that is, instead of encrypting messages that only the holder of the Private Key can decrypt, a message is encrypted with the Private Key so that everyone else can decrypt it using the Public Key. In this case, even though the message is encrypted, the word signature is used to represent that it is no secret. The signature itself has the same format as an encrypted message, it is a sequence of bytes.

This scheme does not provide privacy, since everyone can decrypt the signature, but it provides Authentication (Only the holder of the Private Key could have signed this message) and in some context also legal Non-Repudiation. Some countries consider that digitally signed files/messages are subject to non-repudiation.

The Key Pair is exactly the the same as the one used in Public Key Encryption, however the terminology changes to avoid confusion. If Public Key Encryption is an asymmetric analog as symmetric encryption, digital signatures can be thought of as an asymmetric analog of HMACs.

Public Key Encryption != Certificate Based Communication#

One way to implement Public Key Encryption is through Certificates, a certificate is something bigger in scope than the Key Pair because the Key Pair is simply two sequences of bytes whereas the certificate includes not only the public key but also information about the algorithm, versioning, subject and issuer information and so on.

Therefore, a Key Pair is enough to encrypt and decrypt but a certificate has metadata and additional information to establish a secure channel of communication. One of the most widely used algorithms for Public Key Encryption is RSA and the current standard for certificate is X.509 (which uses RSA Public Key Encryption). Certificate based communication is used in most website, the key indicator is the use of TLS, more commonly reflected by the use of HTTPS (instead of HTTP).

Some of the information a certificate contains is:

Information about the identity (Email, Organization Name, Country, State, among others) of who generated the Public Key (the subject).
A digital signature 1 that validates the identity information is correct and it corresponds to the subject.

The drawback of this approach is that nothing impedes an attacker to sign their own certificates, i.e. anyone can claim being anyone else. These are the so-called self-signed certificates. To avoid that, the signature should come from a trusted third party, known as Certificate Authority. It needs to be an independent and trusted-by-everyone party that validates the identity of who is generating the Public Key. This is necessary if the subject is unknown or could not be trusted.

When the subject acts as its own CA, generating a self-signed certificate, it triggers warnings in most environments, e.g. most programming libraries will throw validation errors. Web browsers will consider a self-signed certificate to be insecure. It is not a matter of security (i.e. the data will be encrypted anyway) but rather a matter of trust, trusting that the received Public Key comes from the intended subject and that there is no other man-in-the-middle.

For debugging and testing, self-signed certificates are usually not a concern, for all other use cases, a certificate signed by a CA should be used.

To get a certificate signed by a CA, one has to submit a Certificate Signing Request, it normally takes several days and it is usually paid service, they also validate that the identity information corresponds to the one asking for the validation. A CA will not sign certificates to anyone on behalf of anyone else, i.e. I cannot get a CA signing a certificate saying I am Google.

Example#

When logging in a service (e.g. email), one enters a username and a password. It is desired that the password travels encrypted through internet, so that no one can read it but the service provider. To encrypt that, certificate based communication is used, the Public Key is used to encrypt the password and then the service provider can use its private key to read it.

That being said, what happens if the Public Key used does not come from the service provider and instead was injected by an attacker’s Key Pair? They would have a corresponding Private Key with which they can see the password (and then if needed redirect to the real service provider).

That is when Certificate Authorities come in, because the certificate will also include a digital signature saying “This is the public key for this service provider”. Attackers can mimic Public Keys from any subject, however, they cannot bypass the digital signature, because everyone can verify if a digital signature comes from a CA or not.

Other usage for Certificates#

Another usage of certificates is User Authentication, in this case one organization can generate specific certificates for each user with all the authentication relevant information. The user can add that certificate to their operating system, and then when making a request only a user id (email, GUID or similar) should be provided.

If the user id matches the information in the certificate and the signature of the certificate is valid (i.e. is was signed by the organization server), the user is consider authenticated and the request is processed.

An example of such authentication mechanism in Flask can be seen in this Anaconda Repo

Practical Example#

This particular site has HTTPS with a certificate signed by DigiCert, one of many Certificate Authorities.

When opened, some details are shown

However, something might draw special attention, it says “Issued to: www.github.com” but the site is elc.github.io. If we inspect the details, we can see that the Issuer is the CA, the Subject is Github and the there is an special field called Subject Alternative Name, there one of the DNS Names is *.github.io which is compatible which this site URL. Therefore, the browser knows the certificate is from Github, validated by DigiCert and even though the URL is not Github’s it is under one of the registered alternative names.

Using Certificates - Implementation Considerations#

Some times the process of getting a signed certificate might be troublesome or tedious, fortunately there are shortcuts:

Using a hosting service that provides HTTPS out of the box (e.g. Github Pages does it freely)
Outsourcing the certificate request (e.g. Most cloud providers will do that on our behalf)
Using hosted services (e.g. Azure Web Apps comes with HTTPS support)

It is usually useful to work in layers, many times one can have a gateway server (Apache, Nginx or similar) that takes care of the TLS connection (certificate based communications) while the underlying application use it seemlessly. Meaning there is no code change needed on the application side.

The following examples will focus on the Public Key Encryption without the CA signing process for simplicity. To implement a certificate signed by a CA, follow the official tutorial. However, unless being supported by a security expert it is always recommended to trust hosted/managed services instead of doing security from scratch.

Caveats of Public Key Encryption#

Asymmetric Encryption is usually much slower than symmetric encryption, that is way in many applications both are used:

Both parties exchange their Public Keys.
The Public Keys are then used to send an encrypted password/key for symmetric encryption.
All following communication is done encrypted using the symmetric encryption method.

The particular details of the implementation may vary depending on the specific protocol but generally Asymmetric Encryption is used as the so-called “Secure Channel” to exchange the keys for the symmetric encryption method. Modern HTTPS connections work like this.

RSA Encryption#

RSA is one of the asymmetric encryption algorithms available in the PyCA cryptography library. The particular objects used here are part of the hazmat package, hazmat stands for “Hazardous Materials” and quoting from their site:

This is a “Hazardous Materials” module. You should ONLY use it if you’re 100% absolutely sure that you know what you’re doing because this module is full of land mines, dragons, and dinosaurs with laser guns.

from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import rsa, padding
from cryptography.exceptions import InvalidSignature

Generating Keys#

The first step is to generate the private key, for that two parameters are needed the public_exponent and the key_size. The former should be fixed to 65537 whereas the second can be changed and as per modern security standards it should be at least 2048. Then the Public Key is generated from the private Key object.

def generate_key_pair():
    key_size = 2048  # Should be at least 2048

    private_key = rsa.generate_private_key(
        public_exponent=65537,  # Do not change
        key_size=key_size,
    )

    public_key = private_key.public_key()
    return private_key, public_key

private_key, public_key = generate_key_pair()

Encrypting#

For the encryption process only the public key is used, no password is needed. The messages encrypted with this Public Key will only be decryptable with the linked Private Key. Moreover, if other messages are encrypted with different Public Keys, they will not be decryptable with the linked Private Key.

def encrypt(message, public_key):
    return public_key.encrypt(
        message,
        padding.OAEP(
            mgf=padding.MGF1(algorithm=hashes.SHA256()),
            algorithm=hashes.SHA256(),
            label=None
        )
    )

message = b"Hello World!"

message_encrypted = encrypt(message, public_key)

print(f"Encrypted Text: {message_encrypted.hex()}")

Encrypted Text: 0f01e45c304eded2e7c31d2e5b0a378b410d89def4398d9fcef69d6c00d783b32df18c7aa26d4e00acecc95ee2fb82218cac7437ee4dcdd773ff873af5db302dfe7f6b12b6d7d3f23fa61f925b878b31b450746572aeaa3a169e2bdf9eb6f820e7ca0ebb90874f469b19c0280636719f14c885ea6ddf7469384047a6f59f3bd24376588e31aa28067459e53a2e44fc6721fe6a63cb59d7621c6c045990aa9a42bd4756740ee116ae40680e2593109d5748d6c35a246485e36bf282bab35bddc9aa2d427700e1d7f52867d28034b7ba43c00777368571fc79b89a3df807dd9790c29bbee858e7030e78d63b6c3147aad4ddffc32fc5a01bd81082a23526158b6a

Decrypting#

Auxiliary Function to decrypt an encrypted message and returning a string message.

def decrypt(message_encrypted, private_key):
    try:
        message_decrypted = private_key.decrypt(
            message_encrypted,
            padding.OAEP(
                mgf=padding.MGF1(algorithm=hashes.SHA256()),
                algorithm=hashes.SHA256(),
                label=None
            )
        )
        return f"Decrypted Message: {message_decrypted}"
    except ValueError:
        return "Failed to Decrypt"

Using unlinked Private Key#

If other Private Key different from the one of the original Key Pair is used, the decryption will fail.

private_key_other, _ = generate_key_pair()

message_decrypted = decrypt(message_encrypted, private_key_other)
print(message_decrypted)

Failed to Decrypt

Using unlinked Public Key#

If other Public Key different from the one of the original Key Pair is used, the decryption will fail.

_, public_key_other = generate_key_pair()

message_encrypted_other = encrypt(message, public_key_other)

message_decrypted = decrypt(message_encrypted_other, private_key)
print(message_decrypted)

Failed to Decrypt

Using original Private Key#

message_decrypted = decrypt(message_encrypted, private_key)
print(message_decrypted)

Decrypted Message: b'Hello World!'

Signing Message#

To use the Key Pair for Digital Signature, the Private Key is used for encryption. In this case the method is called sign and not encrypt to have consistent terminology and avoid confusion.

A pair (message, signature) will only be valid if the message, the signature and the public key are the original ones, if any of those gets modified, an error will be thrown.

def sign(message, private_key):
    return private_key.sign(
        message,
        padding.PSS(
            mgf=padding.MGF1(hashes.SHA256()),
            salt_length=padding.PSS.MAX_LENGTH
        ),
        hashes.SHA256()
    )

message = b"My website is http://elc.github.io"

signature = sign(message, private_key)

print(f"Digital Signature: {signature.hex()}")

Digital Signature: ae7609815b5ff959aac2f86addeb4eef717cc2b4455d7fd73e8433224f52ecbbcbb527a0dcb73028f9905c9dcf58a866977f7956d571896f1d15729652f2ae9a331d9f61d395d927e8cd563c231aa1456238acad48f18d102f6c7eb20dd1523c63b94b9d7b395d65585159615acff15500916ded87198e250dc8473ca59eebaa8084472fb6e5630b6f808d260c8b5739ebbc87daf92137eae71f0b2c570115da5feeecf0dc5ed6bd2efe88f3c22d411fc49b4571fb9bbab3b86d0430285dcc933421a0c66f66c49592b23dac523052404a5baf93984dc38cc07cb0af575f4a904e4bcbcfb508e0e182d866d38b06b18f5c842124c9889478fce72982703353b5

Verifying Signature#

This is a helper function to avoid code duplication, it will return a string message depending on whether the message, signature pair is valid.

def verify(signature, message, public_key):
    try:
        public_key.verify(
            signature,
            message,
            padding.PSS(
                mgf=padding.MGF1(hashes.SHA256()),
                salt_length=padding.PSS.MAX_LENGTH
            ),
            hashes.SHA256()
        )
        return "The message has been successfully verified"
    except InvalidSignature:
        return "The signature, the message or the Public Key is invalid"

Tampered Message#

Possible scenario:

An attacker intercepts the message
They change the message and leave the signature intact

wrong_message = b"My website is http://www.google.com"

verification_message = verify(signature, wrong_message, public_key)
print(verification_message)

The signature, the message or the Public Key is invalid

Tampered Message and Signature#

Possible scenario:

An attacker intercepts the message
They change the message and generate another signature with their own Private Key

Note: it would be possible to test leaving the message intact and only changing the signature but that has no practical purpose.

wrong_message = b"My website is http://www.google.com"

fake_private_key, _ = generate_key_pair()
fake_signature = sign(wrong_message, fake_private_key)

verification_message = verify(fake_signature, message, public_key)
print(verification_message)

The signature, the message or the Public Key is invalid

Unliked Public Key#

This is an unlikely possibility. Because since Public Keys are shared and public, an attacker has very limited chance to inject a fake public key. It would need to hack the sender and modify their Public Key, if they were able to do such a thing, they would have access the original message anyway.

However, for illustration, if the Public Key is changed, the validation fails as well.

_, public_key_other = generate_key_pair()

verification_message = verify(signature, message, public_key_other)
print(verification_message)

The signature, the message or the Public Key is invalid

Original Message, Signature and Public Key#

Only when the message, the signature and the public match will the verficiation process succeed

verification_message = verify(signature, message, public_key)
print(verification_message)

The message has been successfully verified

Using PEM Files#

Public Keys, Private Keys and certificates can be saved as files, in that case the Privacy-Enhanced Mail (PEM) format is used. Those are files with suffix .pem, .cer, .cert or .crt. They have a characteristic BEGIN and END line which encloses the content. Since a key pair is a sequence of bytes, base64 is used to convert those to string.

The extension is just to tell the content but the PEM format is a plain-text file, i.e. it can be opened with any text editor.

PEM Files can be used for either Public Key Encryption or Digital Signature.

from pathlib import Path

from cryptography.hazmat.primitives import serialization

Storing the Keys as PEM Files#

Saving Private Key#

The PEM file for the private key should be kept secret and never shared, the whole asymmetric encryption depends on it being hidden.

Because the Public-Key Cryptography Standards (PKCS) #8 (PKCS8) is used as the serialization format, the private key is not stored in raw bytes but rather it is encrypted using symmetric encryption. The symmetric algorithm used is PBKDF2. Therefore, in theory there should not be any risks if the files is leaked, that being said, sharing private key files is against all good practices.

Note: NEVER upload private key files to source control, make sure to add the file to your .gitignore

password = b"my secret"

key_pem_bytes = private_key.private_bytes(
   encoding=serialization.Encoding.PEM,  # PEM Format is specified
   format=serialization.PrivateFormat.PKCS8,
   encryption_algorithm=serialization.BestAvailableEncryption(password),
)

# Filename could be anything
key_pem_path = Path("key.pem")
key_pem_path.write_bytes(key_pem_bytes);

warning_message = "\n\n     TRUNCATED CONTENT TO REMIND THIS SHOULD NOT BE SHARED\n"

content = key_pem_path.read_text()
content = content[:232] + warning_message + content[1597:]

print(content)

Saving Public Key#

The PEM file for the public key will be part of a public certificate which will be send to everyone wanting to comunicate with the subject. There is no risk sharing this.

public_key = private_key.public_key()

public_pem_bytes = public_key.public_bytes(
   encoding=serialization.Encoding.PEM,
   format=serialization.PublicFormat.SubjectPublicKeyInfo,
)

# Filename could be anything
public_pem_path = Path("public.pem")
public_pem_path.write_bytes(public_pem_bytes);

public_key_content = public_pem_path.read_text()
print(public_key_content)

Loading PEM Files#

PEM files can be loaded directly using the load methods, in the case of the private key, a password should be provided because the PKCS8 was used to serialized it.

Wrong Password#

The cryptography library will throw ValueError if the password is incorrect

private_pem_bytes = Path("key.pem").read_bytes()
public_pem_bytes = Path("public.pem").read_bytes()

guess_password = b"my pass"

try:
    private_key_from_pem = serialization.load_pem_private_key(
        private_pem_bytes,
        password=guess_password,
    )
    public_key_from_pem = serialization.load_pem_public_key(public_pem_bytes)
    print("Keys Correctly Loaded")
except ValueError:
    print("Incorrect Password")

Incorrect Password

Right Password#

If the correct password is used, no errors should be thrown

private_pem_bytes = Path("key.pem").read_bytes()
public_pem_bytes = Path("public.pem").read_bytes()

try:
    private_key_from_pem = serialization.load_pem_private_key(
        private_pem_bytes,
        password=password,
    )
    public_key_from_pem = serialization.load_pem_public_key(public_pem_bytes)
    print("Keys Correctly Loaded")
except ValueError:
    print("Incorrect Password")

Keys Correctly Loaded

Encryption and Decryption using PEM Files#

This are simple examples of encryption and decryption, the functions used are the same as the ones in the above section

Encrypting#

message = b"Hello World!"

public_pem_bytes = Path("public.pem").read_bytes()
public_key_from_pem = serialization.load_pem_public_key(public_pem_bytes)

message_encrypted = encrypt(message, public_key_from_pem)

print(f"Encrypted Text: {message_encrypted.hex()}")

Encrypted Text: b58f3e7cc8c40d5dec6f2509e602d785fbd90a97f2605c1cc443068641c2cc3f06178a90995a2b20d7c8a6e1a9f78021b497cac57fb1f0c02d518783a5c77d573137dbd9875b8cae59c3fc94c6553c79bf18712ee9d7885f73124942fd1a097421d5b2ec1c46033e988e4d4db710bef2bac26a5aec1a9003ec167ee6c08bf5e30e41930d823cb985b5b67873e33cadc50d4cd3aad0b937f46528d0f000d822e1b1e683ff00485f6ae97b082df6824d05b5203097a6f4e3c74838436f39be035752ccc22de9a432467db062c9bdb5983fe2a930e143907643fb153266029d7f9f58dbb026ede1ba51e61838359bb80737e6b73f206e74afc2811d141ca3972009

Decrypting#

private_pem_bytes = Path("key.pem").read_bytes()

private_key_from_pem = serialization.load_pem_private_key(
    private_pem_bytes,
    password=password,
)

message_decrypted = decrypt(message_encrypted, private_key_from_pem)

print(message_decrypted)

Decrypted Message: b'Hello World!'

Digital Signature using PEM Files#

This are simple examples of signing and verification, the functions used are the same as the ones in the above section

Signing a Message#

message = b"My website is http://elc.github.io"

private_pem_bytes = Path("key.pem").read_bytes()

private_key_from_pem = serialization.load_pem_private_key(
    private_pem_bytes,
    password=password,
)

signature = sign(message, private_key_from_pem)

print(f"Digital Signature: {signature.hex()}")

Digital Signature: 2dfa1ee20a201f382eb69b5b0b4b7901330de4eaf6cabdc4926700bb114e39934b1c4feb770ced781a86113f1693130caecee53cc9e422a8fd9830d69c3ae8de9cb3140c1cb00ba28fa87a284382ef6000d1929c072435bc25ce442a6bdf6bcbd87c0b2877b6adda9509987319267265c5a0e7570b5176f6843f3cf15d671c0b2ca145647aa029d329830a8e7cf6be8e2eb9b1de9a141f3547ab513973be194effdf46826f6a30cd14782b1f0925fb453e5abc9647f29a7e9896542c316a740b6f4e636c2d99ccaa01da9291430653c6be8023a0fb4d30cbd047483885584dea246e0e0130d490c8f6d27cb0baa839ba672a320734f262fde6aceae1a4aff127

Verifying Signature#

public_pem_bytes = Path("public.pem").read_bytes()
public_key_from_pem = serialization.load_pem_public_key(public_pem_bytes)

verification_message = verify(signature, message, public_key_from_pem)
print(verification_message)

The message has been successfully verified

Conclusion#

As opposed to symmetric encryption, asymmetric encryption does not rely on a single key shared across parties. Instead, a Key Pair, consisting of a Public and a Private Key which are mathematically linked, is generated by each party, then the public keys are shared. The public key is used to encrypt the message and only the linked Private Key can decrypt it.

Digital Signatures used the same principle in reverse, the Private Key is used to encrypt and the Public Key to decrypt, this does not provide privacy but guarantees that a message has not been tampered with.

For serialization and persistance Key Pairs can be stored in disk using the PEM format, for security reasons the Private Key PEM file is saved encrypted with symmetric encryption and hence requires a passphrase.

Asymmetric Encryption is mostly used as part of a bigger technology called Certificates which provides not only access to the Public Key needed to encrypt the message but also information about the subject, the issuer and a digital signature (among other fields). Certificates could be self-signed (insecure or only suitable for testing) or signed by a trusted third party called Certificate Authority.

The most widely used algorithm for Public Key Encryption and Digital Signatures is RSA and for Certificate Based Comunication is X.509

1: Digital signature is the topic of the next chapter

Python Security

Asymmetric Encryption

Contents

Asymmetric Encryption#

Public Key Encryption != Digital Signature#

Public Key Encryption != Certificate Based Communication#

Example#

Other usage for Certificates#

Practical Example#

Using Certificates - Implementation Considerations#

Caveats of Public Key Encryption#

RSA Encryption#

Generating Keys#

Encrypting#

Decrypting#

Using unlinked Private Key#

Using unlinked Public Key#

Using original Private Key#

Signing Message#

Verifying Signature#

Tampered Message#

Tampered Message and Signature#

Unliked Public Key#

Original Message, Signature and Public Key#

Using PEM Files#

Storing the Keys as PEM Files#

Saving Private Key#

Saving Public Key#

Loading PEM Files#

Wrong Password#

Right Password#

Encryption and Decryption using PEM Files#

Encrypting#

Decrypting#

Digital Signature using PEM Files#

Signing a Message#

Verifying Signature#

Conclusion#