Publish testing packages.

dumol commented 3 years ago

Scope

Upload testing packages to fix #9.

Changes

Used public SFTPPlus instance to add an HTTPS service named Bin Chevah, using a bin user rooted in /srv/sftpplus-bin, with the following rights: allow-read, allow-list, allow-traverse, allow-create-folder, allow-write, allow-rename.

A batch file is generated after building the dist file, to be uploaded through OpenSSH's sftp. New sub-dirs (such as ubuntu2204/x64) are created as needed.

Drive-by changes:

Updated MACOSX_DEPLOYMENT_TARGET to 10.13 to match CFLAGS.
Use the standard short git ID for packages.
Added .gitattributes to fix #3.
Added ngrok-based debugging for Windows runners.
Removed malfunctioning skip-ci steps.
Removed the uploading of artifacts.

Testing

Please review changes.

Check the automated tests.

Download testing packages from https://bin.chevah.com:20443/testing/python/.

Use them in brink.conf to get a Python 3 runtime?

adiroiban commented 3 years ago

But is this a real problem? :-] If yes, we could easily fix it with a repo secret…

Well... we used the different port for security by obscurity ... if we make the port public we are no longer "protected" by obscurity :)

Great that this was moved to SFTPPlus :)

dumol commented 3 years ago

Well... we used the different port for security by obscurity ... if we make the port public we are no longer "protected" by obscurity :)

Yes, there's a bit of security through obscurity. But for me changing default SSH port is more of a thing of convenience. The main purpose being to substantially reduce the number of probes that pollute SSH logs.

dumol commented 3 years ago

For some reason, packages uploaded through GitHub actions are not using the latest commit id…

E.g. for the current last commit, instead of naming the package python-3.8.6.503ec187-ubuntu2004-x64.tar.gz, it is named python-3.8.6.30429be3-ubuntu2004-x64.tar.gz.

Any ideas?

dumol commented 3 years ago

Hhmm, this is still present: Warning: Permanently added the RSA host key for IP address '209.250.251.12' to the list of known hosts.

dumol commented 3 years ago

No more warnings from SFTP through including the host fingerprint for the current IP for bin.chevah.com. Turned on StrictHostKeyChecking too while at it.

Also, git commits are the same no matter if you build the package locally or through GitHub Actions. However, bumping for the nth time into the 8 vs. 7 digits for the short git ID, I have reverted to the standard short 7 digits one. This also fits with what you see in GitHub's web interface.

dumol commented 3 years ago

Surprisingly for how little changes there are, this was quite hard to get right, as you can guess from the amount of commits above. :-]

dumol commented 3 years ago

Maybe also enable full directory (or at least traversal) for https://bin.chevah.com:20443/testing/python/ and https://bin.chevah.com:20443/testing/python/ubuntu2004

allow_traverse is enabled for user bin, as documented in the PR text. What exactly is to be improved?

we also need to enable a read-only theme for the public account... but this is something for salt-master ...and maybe improve the configuration in SFTPPlus

What is a read-only theme? And why is that a problem? (Again, honest questions.)

[…]

I'll add an issue for publishing production packages. Don't know what would be best at the moment…

adiroiban commented 3 years ago

https://bin.chevah.com:20443/testing/python/ubuntu2004

When I go there, I get an empty page and not the listing with x64 folder visible.

What is a read-only theme? And why is that a problem?

A read-only theme as a GUI theme in which all the delete file / upload file / create folder buttons are hidden. It's not a big problem... but those buttons have no usage...and we have customers asking for this feature.

Thanks for the update. Nice work :)

dumol commented 3 years ago

https://bin.chevah.com:20443/testing/python/ubuntu2004

When I go there, I get an empty page and not the listing with x64 folder visible.

OK, I see the difference now, thanks! But what permissions would allow that to work alongside https://bin.chevah.com:20443/testing/python/ubuntu2004/ (with an ending /)?

Current list is allow-read, allow-list, allow-traverse, allow-create-folder, allow-write, allow-rename. The remaining ones seem unrelated, except maybe allow-full-control, which is out of the question.

On a related note, I've noticed files can be overwritten by user bin over SFTP. So I'm not sure in what way they are protected by excluding allow-delete-file anyway… :-]

What is a read-only theme? And why is that a problem?

A read-only theme as a GUI theme in which all the delete file / upload file / create folder buttons are hidden. It's not a big problem... but those buttons have no usage...and we have customers asking for this feature.

OK, I've realized this also at some point yesterday. It doesn't bother me, specifically because the main use case is for the files to be downloaded with wget/curl.

But yes, it would be nice if this would work akin to FTP anonymous access or a simple HTTP listing of files. Now you are shown as Logged in as bin and capable of uploading stuff when simply accessing https://bin.chevah.com:20443/testing/python/ubuntu2004/.

Combining this with the "related note" above, you can anonymously overwrite any file by simply uploading another file with the same name.

adiroiban commented 3 years ago

OK, I see the difference now, thanks! But what permissions would allow that to work alongside https://bin.chevah.com:20443/testing/python/ubuntu2004/ (with an ending /)?

Good question.... I don't think there is any... so this can be a feature request.

On a related note, I've noticed files can be overwritten by user bin over SFTP. So I'm not sure in what way they are protected by excluding allow-delete-file anyway… :-]

Yes. We need a new allow-create permission that will allow writing new files only if they don't exists...and also allow renaming then,

chevah / pythia