Funky Cloud Medina
Home Browse Study Guides About Contact

Download a private GitHub Repo using AWS Lambda

| 1 minute
AWS Hugo GitHub Lambda S3 Blog Python API Gateway How-To

As mentioned in this post I needed to download a private GitHub repo as a ZIP file. The GitHub repo containing this blog is now set to private (mainly to protect my backlog of drafts), which broke my AWS Lambda function (see here and here).

Minimal changes were required from the functioning code, but there was a lot of testing. Some key changes were:

  • Setting the GitHub Personal Access Token as a Lambda variable
  • Modify the file download function to use custom headers
  • Consuming the token from within Lambda to access and download the file

The new function to download the repo:

def downloadSite(account, repo):
    logger.info("Downloading master zip of " + repo + " from GitHub")

    url = 'https://api.github.com/repos/' + account + '/' + repo + '/zipball/master'
    siteZip = "/tmp/master.zip"

    # Download the master repo as a ZIP file
    http = urllib3.PoolManager()
    r = http.request('GET', url, preload_content=False, headers={'Authorization': "token " + os.environ['GITHUB_ACCESS_KEY'],'User-Agent': os.environ['GITHUB_ACCOUNT']})
    with open(siteZip, 'wb') as out:
        while True:
            data = r.read(64)
            if not data:
                break
            out.write(data)
    r.release_conn()
Share this on:
About Stellios Williams
Technical Account Manager VMware
This is my personal tech related blog for anything private and public cloud - including homelabs! My postings are my own and don’t necessarily represent VMware’s positions, strategies or opinions. Any technical guidance or advice is given without warranty or consideration for your unique issues or circumstances.
Contact
Comments
comments powered by Disqus
Latest Posts
Advertisement
Popular Tags
vmware44 how-to17 vrealize-suite16 vsphere13 vcloud-director12 aws8 certificates8 professional-development7 vrealize-orchestrator7 github6 hugo6 lambda6 vcenter6 blog5 s35 python4 vrealize-suite-lifecycle-manager4 api3 api-gateway3 automation3 esxi3 homelab3 javascript3 psc3 cassandradb2 certification2 hardware2 powercli2 rest2 storage2 swagger2 vcloud-availability2 vcsa2 vrealize-automation2 vrealize-log-insight2 vrealize-operations-manager2 active-directory1 adfs1 azure1 blueprints1 bmc1 certificate-authority1 clarityui1 converter1 cpu1 custom-resource1 dell1 dns1 docker1 draas1 federation1 firmware1 freebsd1 freenas1 gcp1 ipmi1 json1 microsoft1 network1 nodejs1 nsx1 opensource1 packer1 pfsense1 postman1 quanta1 route531 saml1 self-hosted1 upgrade1 vcloud1 vexpert1 vrealize-operations1 vro1 vsan1 vvols1 windows-server1
Blogroll