Integrate with GitHub

[obbardc]

pdk push and pdk pull commands have been disabled.

Would be nice to enable these, so we can keep track of distro changes in remote repositories.

Also, can we easily make these commands work with GitHub repos? Shouldn't be too hard.

[obbardc]

The etc/channels.xml file is needed by PDK, and should be under version control. I have made changes to PDK to allow files inside etc/ to be under version control.

This commit also ignores files under etc/tmp and tmp. I think we will use the etc/tmp folder for temporary files (i.e whilst generating images), so we may remove tmp from the ignore file in the future. commit #2c6c8559bd1007416e74d40f09f2608eb0b1944f

I have added support for local .gitignore file in the workspace. The reason I decided to call the file this rather than ignore or whatever is so that all of the standard git commands will still work. commit #88f6bdbf7567bfa156c599fc2ba5dc09897cf9c7 commit #15c8c3cb49e85ddc4edc3a058188cfe61dfe604f

[obbardc]

For now we can do this using basic Git pull and push commands. But I think a better way of storing components will be using a plain old Git repository, rather than having to use any HTTP server in the middle.

The advantage to this is not having to store any files on any of our severs and the expensive bandwidth requirements of doing so.

I'm thinking we store all of the component XML files, and any other Deb files needed, on GitHub. The pull/push commands will look something like this: pdk pull origin and pdk push origin will pull & push to the remote Git repo, if setup in the components.xml file. You will then be asked for Username & Password just as if you were pushing using the git push command.

pdk pull <component_name> will just pull the HEAD of the component in question and merge into the HEAD of the local repository.

The only thing that worries me doing this is that blobs are not stored in the PDK git repository. Where do we store our custom Debs which are required for build? I guess we could host an APT repo...... Which is what I guess we have done in the past with apps like the old cdd-utils, backported debs etc.

Let me know what you think. As I say this is a future requirement.

    def _anon_http_pull_strategy(self):
        '''Run a pull.

        This method is run when the path indicates that we need to do
        an anonymous http pull.
        '''
        try:
            schema_url = self.full_path + '/etc/schema'
            schema_number = \
                int(get_remote_file_as_string(schema_url).strip())
        except ValueError:
            message = "Remote workspace has invalid schema number."
            raise SemanticError, message
        if schema_number != schema_target:
            raise SemanticError, 'Workspace schema mismatch with remote.'

        blob_list_url = self.full_path + '/etc/cache/blob_list.gz'
        get_remote_file(blob_list_url, self.channel.channel_file)
        self.world.index_world_data()

        git_path = self.full_path + '/etc/git'
        self.vc.direct_pull(git_path, self.upstream_name)

[danielhjames]

According to http://www.hackgnar.com/2016/01/creating-remote-apt-package.html it is possible to host an APT repo directly in a GitHub project by making use of the raw.githubusercontent.com URLs.

[obbardc]

@danielhjames Right, I did think about raw file access, but I would like to not restrict it to GitHub if possible

[obbardc]

Problem with this is that all debs are stored in PDK repos. This is one for the far_future!

[danielhjames]

Another option is https://packagecloud.io/ but it's at their discretion whether to host an open source project for free. Might be useful for commercial PDK users.

[obbardc]

The problem is the cache folder, it stores all debs as blobs so really has to be pulled over HTTP.

For now, I think it is OK as we can still back-up & share PDK project sources on Github by adding another Git remote :-).

git remote add github git push github master

Cheers!

Christopher Obbard Chief Engineer 64 Studio Ltd.

On 29 May 2018 at 10:24, Daniel James notifications@github.com wrote:

Another option is https://packagecloud.io/ but it's at their discretion whether to host an open source project for free. Might be useful for commercial PDK users.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/64studio/pdk/issues/19#issuecomment-392711799, or mute the thread https://github.com/notifications/unsubscribe-auth/AA-11t8R8Mgeu7Q5mk0v4pRQwiPFpAnkks5t3RPlgaJpZM4J9SpY .