klmhsb42 commented 4 years ago

Would be nice to have paragraph "IP addresses" with info:

That it is to indentify a device
That there are IPv4 and v6
That there are local and public
Where to find IP address of device
That there are IPv4 only, dual stack (with both), dual stack lite (access IPv6 only), and IPv6 only
How to know which of those systems is used

klmhsb42 commented 4 years ago

Could you also explain your App porting guide a bit more?

If I fork an app from your repo, do I still have to Generate .drone.yml from .drone.jsonnet? And should I do that on Syncloud device or before on my linux computer? And if I have a branch with changes in my repo and I want to install that, can I run snap install from github URL? And how?

cyberb commented 4 years ago

If I fork an app from your repo, do I still have to Generate .drone.yml from .drone.jsonnet?

Only if you modified it.

And should I do that on Syncloud device or before on my linux computer?

It is much more convenient to do it on a PC with all the tools you like installed.

And if I have a branch with changes in my repo and I want to install that, can I run snap install from github URL?

You need to build a package (my_app.snao) which then you need to copy to a device and install. GitHub is for sources and for image distributions. Apps when they are ready for public stored in Syncloud Store (apps.syncloud.org)

klmhsb42 commented 4 years ago

You could also add a hint to VirtualBox section, that IPv6 on VM image doesn't work with WiFi of host. There is an issue https://www.virtualbox.org/ticket/5503 which was closed, but the bug is still there...

klmhsb42 commented 4 years ago

491

klmhsb42 commented 4 years ago

470

cyberb commented 4 years ago

Added more sections to the porting guide.

If I fork an app from your repo, do I still have to Generate .drone.yml from .drone.jsonnet?

If you do not change jsonnet you do not need to regenerate yml.

And should I do that on Syncloud device or before on my linux computer?

Building happens on any Linux pc.

And if I have a branch with changes in my repo and I want to install that, can I run snap install from github URL? And how?

Snap does not install what we have in git it installes a package which we produce from git. Git does not contain any binary, it is a source which we compile and download other deps during the build. So when you have built a package you can copy it to a device and there install it.

klmhsb42 commented 4 years ago

Could you add a section "Copy snap to device"

And if I have my snap package, which directory in syncloud root would be the best to move the snap package there for installation (as I can't install from app center). I've seen that I don't have always write rights even as root per ssh.

cyberb commented 4 years ago

You could also add a hint to VirtualBox section, that IPv6 on VM image doesn't work with WiFi of host. There is an issue https://www.virtualbox.org/ticket/5503 which was closed, but the bug is still there...

This will have to be tested on the upcoming buster based image

cyberb commented 4 years ago

Could you add a section "Copy snap to device"

And if I have my snap package, which directory in syncloud root would be the best to move the snap package there for installation (as I can't install from app center). I've seen that I don't have always write rights even as root per ssh.

You need to use scp and by default it is home of the target user unless specified explicitly.

pc# scp platform-123.snap root@device
pc# ssh root@device
device# snap refresh platform-123.snap --devmode

klmhsb42 commented 4 years ago

If I fork an app from your repo, do I still have to Generate .drone.yml from .drone.jsonnet?

If you do not change jsonnet you do not need to regenerate yml.

If I run arch=[amd64|arm] /path/to/cli/.drone.yml exec --trusted I get bash: arm]: command not found

I guess this will create package.snap?

klmhsb42 commented 4 years ago

Note for me: /path/to/cli/drone means /path/to/cli/.drone.yml

cyberb commented 4 years ago

Note for me: /path/to/cli/drone means /path/to/cli/.drone.yml

No it is not, it is path to drone cli (command line interface) binary.

➜  snapd git:(master) ../drone --help
NAME:
   drone - command line utility

USAGE:
   drone [global options] command [command options] [arguments...]

VERSION:
   1.2.1

COMMANDS:
     build      manage builds
     cron       manage cron jobs
     log        manage logs
     encrypt    encrypt a secret
     exec       execute a local build
     info       show information about the current user
     repo       manage repositories
     user       manage users
     secret     manage secrets
     server     manage servers
     queue      queue operations
     orgsecret  manage organization secrets
     autoscale  manage autoscaling
     fmt        format the yaml file
     convert    convert legacy format
     lint       lint the yaml file
     sign       sign the yaml file
     jsonnet    generate .drone.yml from jsonnet
     starlark   generate .drone.yml from starlark
     plugins    plugin helper functions
     help, h    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   -t value, --token value   server auth token [$DRONE_TOKEN]
   -s value, --server value  server address [$DRONE_SERVER]
   --autoscaler value        autoscaler address [$DRONE_AUTOSCALER]
   --help, -h                show help
   --version, -v             print the version

Here is how to run it, assuming you are inside a project dir where you have .drone.jsonnet file which will be converted to a .drone.yml

➜  snapd git:(master) ../drone jsonnet --stream

The next step is to run drone exec to produce a snap package.

klmhsb42 commented 4 years ago

Ok, do I need local installation of snapd from your fork https://github.com/syncloud/snapd first? I can download this repo and what do I have to run then for installation? Or is original snapd fine?

cyberb commented 4 years ago

Building snap package does not require snapd. While installation onto a device requires Syncloud Snapd. This repo is a source repo (snapd is written on go language) and requires compilation. So the easiest option is of cause use an existing Syncloud device or VM to test your new snap package.

I do know how much of development experience you have, I think there is a confusion between source repo and binary packages. Languages are devided into compiled (go, c/c++, rust, java ...) and not-compiled (python, ruby, php, js, ...). Also packaging regardless of compilation involves preparation of a various pieces of project into a ready to use archive (zip, tar.gz, snap)

klmhsb42 commented 4 years ago

I think there is a confusion between source repo and binary packages. Languages are devided into compiled

You are right, I will keep that in mind and read about it.

It worked now, as I was running drone jsonnet --stream inside my directory. My mistake was to run it without drone command as I had only the path /drone.

The next step is to run drone exec to produce a snap package.

I get for drone exec --trusted this error: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? So I have to install docker first?

klmhsb42 commented 4 years ago

So I have to install docker first?

I did, and if I run drone exec --trusted in the same directory, this makes error: [device:0] standard_init_linux.go:211: exec user process caused "exec format error"

klmhsb42 commented 4 years ago

[device:0] standard_init_linux.go:211: exec user process caused "exec format error" [version:0] standard_init_linux.go:211: exec user process caused "exec format error" [artifact:0] 2020/04/19 16:54:18 Error: missing server host 2020/04/19 18:54:19 artifact : exit code 1

klmhsb42 commented 4 years ago

Should Docker emulate the Arm architecture or should I change the base image to x86? I will need it on the device as arm based image, right?

drone arch=[amd64|arm] exec --trusted or arch=[amd64|arm] drone exec --trusted leads to bash: arm]: command not found

klmhsb42 commented 4 years ago

My local machine is x86/64.

klmhsb42 commented 4 years ago

Do I need to install QEMU and copy the interpreter to the container?

cyberb commented 4 years ago

Right, arm packages should be built on arm devices, amd64 on amd64(x64). I forgot about it, as I have arm and x64 build agents. So you can either do run all this on your arm syncloud device (install drone and docker) or on your x64 pc but then test the package on VirtualBox with x64 Syncloud image.

Arch parameter should always be the same as host arch:

x64: drone arch=amd64 exec --trusted Arm: drone arch=arm exec --trusted

Sorry it may feel very complex but it will be simpler if you know all the steps :) of cause I am open to any simplification if there is any.

cyberb commented 4 years ago

I did not really bother with cross-compilation where you can build arm on x64 and other way around as it is slow and not very simple so as I have many arm and x86 devices I decided just to build packages on their corresponding native architectures and that works pretty good.

klmhsb42 commented 4 years ago

drone arch=amd64 exec --trusted Is this the exact command I should use (when I'm inside the directory)? I get:

No help topic for 'arch=amd64'

With arch=amd64 drone exec --trusted it runs for a while and I get then:

[device:0] standard_init_linux.go:211: exec user process caused "no such file or directory" [version:0] standard_init_linux.go:211: exec user process caused "no such file or directory" [artifact:0] 2020/04/20 15:43:45 Error: missing server host 2020/04/20 17:43:45 artifact : exit code 1 Not sure where to put arch=amd64 as this is not a drone command I guess.

And where should the snap file be saved? Just one level lower in the directory or somewhere else?

it will be simpler if you know all the steps :)

I'm close getting to it ;)

klmhsb42 commented 4 years ago

with DOCKER_API_VERSION=1.24 arch=amd64 installer=sam exec I get no error message, but watch -n 1 pstree -a $(pgrep -f dockerd) shows also nothing and I can't find any snap file...

klmhsb42 commented 4 years ago

Also just arch=amd64 exec runs without error, but I can't see any output

klmhsb42 commented 4 years ago

Output should be [appname].yaml right?

klmhsb42 commented 4 years ago

Output should be [appname].yaml right? Ok, no, should be .snap

cyberb commented 4 years ago

which project are you running this in? what is your .drone.jsonnet/.drone.yml? can you show drone log?

cyberb commented 4 years ago

Sorry since drone v1.0 the command is different (updated the guide):

sudo ../drone/drone exec --pipeline=amd64 --trusted

Ideally you only need two steps (version and build) and drone supposed to handle this with --include flag but it does not like more than one at the moment

sudo ../drone/drone exec --include=version,build --pipeline=amd64 --trusted

Question: https://discourse.drone.io/t/exec-include-does-not-work-with-more-than-one-step/7186

klmhsb42 commented 4 years ago

sudo ../drone/drone exec --pipeline=amd64 --trusted

It work!!! 👍 Only the testing at the end was stopped, as I guess it tries to connect per ssh to a device?

pc# scp platform-123.snap root@device
pc# ssh root@device
device# snap refresh platform-123.snap --devmode

Did not work yet. It creates a Impulse Tracker audio (audio/x-it) file with a name of the ssh host. Can't find the snap at the device.

exec --include=version,build

What's the advantage/difference of this?

One thing which could be improved, if you run the drone exec, that it could overwrite 3rdparty packages that you do not mess up the directory, if you do it several times... What do you think?

cyberb commented 4 years ago

Did not work yet. It creates a Impulse Tracker audio (audio/x-it) file with a name of the ssh host. Can't find the snap at the device.

What do you mean? Could you show the output of each of the three commands?

What's the advantage/difference of this?

The advantage is to not run steps you do not have access to, like upload, artifacts ...

klmhsb42 commented 4 years ago

What do you mean?

The moment, when you should take a break :) scp platform-123.snap root@device command does not print anything. Normally it should ask for a password like ssh command, right? It does not ask or print or wait for anything. You can type in next command immediately. It creates only a file with the name root@device.syncloud.it (recognized from linux as Impulse Tracker audio (audio/x-it) file format). Do I need to install scp, when I have ssh working? Is there a way to watch copy process? It will also take time, right?

The advantage is to not run steps you do not have access to, like upload, artifacts ...

Makes sense...

klmhsb42 commented 4 years ago

Anyway, I can also copy by remote access. I downloaded files repo from github, wrote "test" into <h2> html tag in login.html, exec the .yml and installed the snap file. Installation was without any error message, but if I open the domain it shows nginx server - 404 Not Found. The app is shown in device web ui under apps, but the icon is not there...

cyberb commented 4 years ago

scp platform-123.snap root@device command does not print anything

sorry, correct command is to use colon at the end:

scp platform-123.snap root@device:

klmhsb42 commented 4 years ago

scp platform-123.snap root@device:

Works!

shows nginx server - 404 Not Found

Which log would make most sense to show you?

cyberb commented 4 years ago

snap list

klmhsb42 commented 4 years ago

Name       Version    Rev        Tracking  Developer  Notes
files      200422     x1         -         -          devmode
gogs       19080616   19080616   stable    syncloud   -
nextcloud  200228131  200228131  stable    syncloud   -
openvpn    20012589   20012589   stable    syncloud   -
platform   200413267  200413267  stable    syncloud   base

cyberb commented 4 years ago

tail -100 /var/snap/platform/common/log/nginx_public_error.log
tail -100 /var/snap/platform/common/log/nginx_public_access.log

klmhsb42 commented 4 years ago

Result per mail. It doesn't find all the files. Do I have to copy only the snap file to device or the complete folder?

cyberb commented 4 years ago

a least this was 200 (OK)

GET /css/lib/bootstrap-3.3.1.min.css HTTP/2.0" 200 113498 "https://files.***.syncloud.it/"

Could you send this?

tail -100 /var/snap/files/common/log/nginx_error.log
tail -100 /var/snap/files/common/log/nginx_access.log

What about removing files and installing from the store?

klmhsb42 commented 4 years ago

Could you send this?

tail -100 /var/snap/files/common/log/nginx_error.log
tail -100 /var/snap/files/common/log/nginx_access.log

See mail...

What about removing files and installing from the store?

That is working.

klmhsb42 commented 4 years ago

In ~/var/snap/files/ there are 3 folders common current x1 which are all empty.

In ~/snap/files/ there are 2 folders current x1 while both contain all folders and files (nginx etc.).

klmhsb42 commented 4 years ago

What's the difference between common, current and [number]?

cyberb commented 4 years ago

snapd keeps 3 versions of an app for rollback.

/var/snap is a writable location for app generated files. /snap is a read-only app files.

current is a link to a specific version. common is a common writable dir to keep data between upgrades.

ls -la /snap/files
ls -la /var/snap/files

Added a link to snapd documentation to the guide. here is the section about it: https://snapcraft.io/docs/environment-variables You are looking for SNAP, SNAP_COMMON, SNAP_DATA

klmhsb42 commented 4 years ago

Added a link to snapd documentation to the guide.

Thanks for the link, I will check it out...

In ~/var/snap/files/ there are 3 folders common current x1 which are all empty.

Do you have any idea, why it's empty and I get error 404?

klmhsb42 commented 4 years ago

/var/snap is a writable location for app generated files.

Or are they only filled with content, if you use the app?

common current x1 folders in ~root/snap/ are empty ,too.

cyberb commented 4 years ago

Why

In ~/var/snap/files

Instead of this

ls -la /var/snap/files

Also try removing your version and install from store and compare.

klmhsb42 commented 4 years ago

Instead of this
ls -la /var/snap/files

root@syncloud:~# ls -la /var/snap/files
total 24
drwxr-xr-x 6 root root 4096 Apr 15 22:52 .
drwxr-xr-x 7 root root 4096 Apr 14 22:53 ..
drwxr-xr-x 6 root root 4096 Apr 15 22:52 common
lrwxrwxrwx 1 root root    2 Apr 15 22:52 current -> x3
drwxr-xr-x 2 root root 4096 Apr 14 22:53 x1
drwxr-xr-x 2 root root 4096 Apr 14 22:53 x2
drwxr-xr-x 2 root root 4096 Apr 14 22:53 x3

Sorry that I tried to install it multiple times, but it would look the same with only x1

Also try removing your version and install from store and compare.

Ok, I will try that...

klmhsb42 commented 4 years ago

Also try removing your version and install from store and compare.

Ok, I will try that...

Ok, they look actually the same, except the icon in https://device.syncloud.it/app.html?app_id=files is missing

klmhsb42 commented 4 years ago

ok, I installed first from store and then from my package without removing previous. Durring installation, output is:

Mount snap "files" (unset)
Stop snap "files" services
Set automatic aliases for snap "files"
Run post-refresh hook of "files" snap if present
Start snap "files" (unset) services
Snap files is no longer tracking stable.

syncloud / platform

Github wiki add content #466

491

470