search

threefoldtech / home

Starting point for the threefoldtech organization
https://threefold.io
Apache License 2.0
9 stars 4 forks source link

building should happen on our TFGrid #718

Closed despiegk closed 4 years ago

despiegk commented 4 years ago
despiegk commented 4 years ago

wasn't there a test space in hub??? where there is no auth? every image > 90 days gets removed

zaibon commented 4 years ago

how is security in the hub now, if we do API approach

At the moment the only way to authenticate to the hub API is using IYO JWT. There is an open issue to support 3bot connect: https://github.com/threefoldtech/0-hub/issues/28 but 3bot connect doesn't provide any way for non interactive authentication at the moment.

wasn't there a test space in hub??? where there is no auth?

There is not hub frontend without authentication. But there is an 0-db that is open for writing for the hub playground: https://playground.hub.grid.tf/. 0-DB address is tcp://playground.hub.grid.tf:9910

Regarding the general idea of building using 0-OS container. I don't see how this could be done. 0-OS container only allow 256MB on the root filesystem. If you need more storage you need to attach a volume to the container. Which means if you need to install extra things to allow the build, this is not going to work cause all installation will end-up on the root filesystem, not on the volume. Working around this would require some bind-mount trick or something. Which are not allowed in container anyhow.

despiegk commented 4 years ago

this is not good I think, so you are telling here that anyone who starts a container who writes more than 256MB breaks the container, not sure this is doable

zaibon commented 4 years ago

Capacity must be paid for. If you do not limit the size of the root filesystem people will abuse the 0-FS cache to store data which is something we must prevent. Also any data that is part of an flist is not counted in the quota. So properly package application will never hit the storage issue.

rkhamis commented 4 years ago

see: https://github.com/threefoldtech/jumpscaleX_threebot/issues/722

zaibon commented 4 years ago

Find a way for people to upload their flist to the hub using API

@maxux What are our options regarding this ? It is possible to create a "guest" user on the playground hub so anyone can push flist there maybe ?

maxux commented 4 years ago

You can now use the token kiUTd9jRjgt7QB6lRh2bcpNiC2UqvTLI to authenticate yourself as guest on the playground server.

This is possible via pull request https://github.com/threefoldtech/0-hub/pull/41

Example using curl:

$ curl -H 'Authorization: bearer kiUTd9jRjgt7QB6lRh2bcpNiC2UqvTLI' https://playground.hub.grid.tf/api/flist/me
{"status": "success", "payload": {"username": "guest"}}

Or via zflist tool:

$ export ZFLIST_HUB_TOKEN=kiUTd9jRjgt7QB6lRh2bcpNiC2UqvTLI
$ ./zflist hub login
[+] initializing zflist
[+] system: entrypoint set from environment variable
[+] system: mountpoint: /tmp/zflistmnt
[+] system: callback found for command: hub
[+] hub: checking authentication
[+] hub: target: https://playground.hub.grid.tf/api/flist/me
[+] hub: sending request
[+] response [200]: {"status": "success", "payload": {"username": "guest"}}
[+] hub: authentication: {"status": "success", "payload": {"username": "guest"}}
[+] hub: authenticated as: guest
[+] hub: authenticated
BolaNasr commented 4 years ago

Flist Manipulation

we using zflist binary in 0-flist repo

to install it using kosmos

 j.builders.storage.zflist.install(reset=True)

create new flist:

create new flist then put file from local then commit it

new_flist = j.data.flist.new() 
new_flist.put("/sandbox/code/github/test.py","/") 
new_flist.commit("/sandbox/code/app.flist") 
#delete everything in temporary-point
new_flist.close()

open flist and edit on it:

open flist , put dir from local , commit it

new_flist = j.data.flist.open("/tmp/app.flist") 
new_flist.put_dir ("/tmp/app","/") 
# list all things in flist
new_flist.list_all()
new_flist.commit("/sandbox/code/app2.flist") 
#delete everything in temporary-point
new_flist.close()

upload flist to hub (Guest):

before upload to hub please export ZFLIST_HUB_TOKEN on this example

new_flist = j.data.flist.open("/tmp/app.flist") 
new_flist.put_dir ("/tmp/app","/") 
# list all things in flist
new_flist.list_all()
new_flist.commit("/sandbox/code/app2.flist") 
new_flist.upload("/sandbox/code/app2.flist")

Then check it at https://playground.hub.grid.tf/guest

Pishoy commented 4 years ago

verified

zflist docs in https://github.com/threefoldtech/jumpscaleX_libs/tree/development/JumpscaleLibs/data/flist

zaibon commented 4 years ago

Before we can close this, proper documentation needs to be put in the sdk manual.

zaibon commented 4 years ago

@maxux is still improving the tooling around this too.

BolaNasr commented 4 years ago

PR: https://github.com/threefoldfoundation/info_tfgridsdk/pull/125