The Python Requests library uses its own CA file by default, or will use the certifi package's certificate bundle if installed. Unfortunately my employer performs SSL interception that re-signs the certificates using their own intermediates, causing errors for external URLs like so:
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
While it's possible to pass your own CA bundle to Requests to override the default CAs,
there's a number of third party packages that use Requests under the hood and no way
to tell them to use a custom location for
verify. In my case, it was the Slackclient
package for working with the Slack Real-Time Messaging API failing to connect when I was
using an internal network. While I could manually update the CA file on my laptop,
deploying the application to our internal platform would install a new Certifi package,
without my updates. Ouch.
However, we can easily check for this when our scripts start up, and update the CA bundle automatically with a given CA if necessary. First, grab your custom CA and save it as a PEM file. If you only have a .crt, .cer or .der file, you can convert it using OpenSSL:
openssl x509 -inform der -in certificate.cer -out certificate.pem
If you have multiple custom roots or intermediates, you can just append them all into a single .pem file when you're finished converting them all.
Drag that certificate.pem into the root of your project. Now, we're going to try making a request to a target URL, and if we hit a cert error, update the CA bundle in use by Certifi:
import certifi import requests try: print('Checking connection to Slack...') test = requests.get('https://api.slack.com') print('Connection to Slack OK.') except requests.exceptions.SSLError as err: print('SSL Error. Adding custom certs to Certifi store...') cafile = certifi.where() with open('certicate.pem', 'rb') as infile: customca = infile.read() with open(cafile, 'ab') as outfile: outfile.write(customca) print('That might have worked.') # Actual app logic here, hopefully without SSL issues.
Success. Or at least, Works On My Cloud. No need to disable SSL verifications, so if the certificate is invalid for reasons other than my network's interception, our application will still fail correctly.