Download a private GitHub Repo using AWS Lambda

| 1 minute
AWS Hugo GitHub Lambda S3 Blog Python API Gateway How-To

As mentioned in this post I needed to download a private GitHub repo as a ZIP file. The GitHub repo containing this blog is now set to private (mainly to protect my backlog of drafts), which broke my AWS Lambda function (see here and here).

Minimal changes were required from the functioning code, but there was a lot of testing. Some key changes were:

  • Setting the GitHub Personal Access Token as a Lambda variable
  • Modify the file download function to use custom headers
  • Consuming the token from within Lambda to access and download the file

The new function to download the repo:

def downloadSite(account, repo):
    logger.info("Downloading master zip of " + repo + " from GitHub")

    url = 'https://api.github.com/repos/' + account + '/' + repo + '/zipball/master'
    siteZip = "/tmp/master.zip"

    # Download the master repo as a ZIP file
    http = urllib3.PoolManager()
    r = http.request('GET', url, preload_content=False, headers={'Authorization': "token " + os.environ['GITHUB_ACCESS_KEY'],'User-Agent': os.environ['GITHUB_ACCOUNT']})
    with open(siteZip, 'wb') as out:
        while True:
            data = r.read(64)
            if not data:
                break
            out.write(data)
    r.release_conn()
Share this on:
About Stellios Williams
Technical Account Manager VMware
This is my personal tech related blog for anything private and public cloud - including homelabs! My postings are my own and don’t necessarily represent VMware’s positions, strategies or opinions. Any technical guidance or advice is given without warranty or consideration for your unique issues or circumstances.
Comments
comments powered by Disqus