Explore how to fetch profile information from a Github repository via Python

[mvlassis]

Context

In order to build the github-profiles-automator charm, we need to be able to fetch information about the wanted profiles from a given Github repository. This task is about the exploration on how that can be implemented in Python.

What needs to get done

1. Look into how to use the proper credentials to fetch information from a requested Github repository.

Definition of Done

1. We know how to write Python code that can properly fetch the profile information from a Github repo, with the correct credentials.

[syncronize-issues-to-jira[bot]]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-6569.

This message was autogenerated

[mvlassis]

After a discussion with @kimwnasptd we have concluded on the following:

• We have decided on using git-sync for our workload container as it fits our usecase exactly.
• git-sync allows us to fetch information from a git repository on a configurable timeout
• We want a way for the workload container to communicate to the charm that the repository has been synced. We believe we can accomplish this using Pebble custom notices.
• git-sync also allows us to specify a command that will run when the timeout is reached. We can configure it to run /charm/bin/pebble notify so that the charm can be notified and run the code we want.
• We still need to figure out how the charm will access the profile information that exists in the workload container.

[mvlassis]

I have created a small demo to demonstrate the feasibility of the workload:

• Created a deployment that uses the upstream git-sync image, which successfully pulls from a public Github repo.
• Pulling from a private repo with an SSH key that has been added to a Github account also works as long as the key itself is not protected with a password, see here.
• Executing any scripts each time new data is fetched also works

Now the remaining parts are:

1. Create a new "demo" charm that has git-sync as a workload container, that can successfully pull from a private repo using SSH keys. As a demo, the charm could hardcode the key, but later we will use juju configs. Additionally, the charm should inject a script named notify-pebble.sh (or something similar) that is a simple wrapper for the Pebble notify event that we will use.
2. Discuss the best way to share the copied data between the workload container and the charm. Write code in the charm that watches for and handles the Pebble notify event. In the final charm, this will call the functions from the library that @kimwnasptd is writing

Thus, we will split this further exploration into 2 tasks.

[mvlassis]

Closing this part, the next parts of the exploration are https://github.com/canonical/github-profiles-automator/issues/5 and https://github.com/canonical/github-profiles-automator/issues/6.