search

melezhik / RakuDist

Test Raku modules against different OS, Rakudo versions
8 stars 0 forks source link

Ability to run on arbitrary rakudo versions #1

Closed AlexDaniel closed 4 years ago

AlexDaniel commented 4 years ago

I propose rakudo_version (or maybe rakudo-version?) parameter for the curl service so that it's possible to test modules on any rakudo version (e.g. rakudo_version=2018.05, rakudo_version=add2ec0d4, etc.). Rakudo builds can be taken from Whateverable so there's no need to waste cpu cycles building rakudo for every request.

This would come handy for rakudo release management as well as for authors so that they can make sure that their modules work on a wide variety of rakudo versions available.

melezhik commented 4 years ago

@AlexDaniel. Are those builds come with zef?

AlexDaniel commented 4 years ago

@melezhik no, it's Rakudo only.

melezhik commented 4 years ago

How do i suppose to build a zef then?

melezhik commented 4 years ago

cc @AlexDaniel

AlexDaniel commented 4 years ago

@melezhik I'm not sure, maybe you'll need to install zef separately (or have it already installed and then just use it with the given rakudo build).

melezhik commented 4 years ago

Yeah. Looks like I've found the way on zef doc. It is possible to build it from a source code.

On Jan 17, 2020 5:59 PM, "Aleks-Daniel Jakimenko-Aleksejev" < notifications@github.com> wrote:

@melezhik https://github.com/melezhik I'm not sure, maybe you'll need to install zef separately (or have it already installed and then just use it with the given rakudo build).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/melezhik/RakuDist/issues/1?email_source=notifications&email_token=AAHRHSICRDXVHK5VJOWQKWDQ6JA5ZA5CNFSM4KIO3GQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJJJ2RA#issuecomment-575839556, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHRHSOECCPBQGBE6OJSURLQ6JA5ZANCNFSM4KIO3GQQ .

melezhik commented 4 years ago

For history, how to install Rakudo from whateverable on docker debian instance:

    9  wget https://whateverable.6lang.org/2019.07.1
   13  apt-get install zstd
   14  zstd -dqc -- 2019.07.1 | tar -x --absolute-names
   18  cd /root/
   19  git clone https://github.com/ugexe/zef.git
   20  cd zef/
   21  /tmp/whateverable/rakudo-moar/40b13322c503808235d9fec782d3767eb8edb899/bin/perl6 -I . bin/zef install .
   29  export PATH=/root/.perl6/bin/:/tmp/whateverable/rakudo-moar/40b13322c503808235d9fec782d3767eb8edb899/bin/:$PATH
   31  zef install --/test Sparrow6
melezhik commented 4 years ago

@AlexDaniel how could I find available commits/versions on whateverable web site, so I could I download and test against different versions

AlexDaniel commented 4 years ago

@melezhik there are builds for all rakudo commits since 2014.

melezhik commented 4 years ago

@AlexDaniel, got you! One more question. It's not clear from whateverable documentation, is it possible to install Rakudo binaries into arbitrary directory, not just /tmp ?

AlexDaniel commented 4 years ago

@melezhik not really. For a long time rakudo wasn't relocatable, so once you built it you couldn't move it. It's different now, and I think you should be able to move it, but you can't do that for older versions (2014-2018 or so).

melezhik commented 4 years ago

yeah, this is what I suspected, anyway, thank you!

melezhik commented 4 years ago

For history:

git clone https://github.com/ugexe/zef.git /root/zef
/tmp/whateverable/rakudo-moar/40b13322c503808235d9fec782d3767eb8edb899/bin/perl6 -I /root/zef/ /root/zef/bin/zef install Kind

This will allow to use zef without installing it, I don't know how to have multiple zefs installed for multiple Rakudos/whateverrables for the same user. This

melezhik commented 4 years ago

@AleksDaniel where can I get rakudo commits history, to experiment with different commits/versions?

AlexDaniel commented 4 years ago

@melezhik https://github.com/rakudo/rakudo/commits/ ?

Alternatively, try to test ADT module on these revisions:

melezhik commented 4 years ago

@AlexDaniel Ok, thanks.

One more question - how to map digital versions into commits? Say I want to know what commit is for 2019.11, the reason I need this know - when a distribution is unpacked it has a commit in folder path, not a version.

melezhik commented 4 years ago

also I noticed that short commits forms still produce long commits forms in /tmp/whateverable/rakudo-moar path,

like add2ec0d4912 build has a add2ec0d491218e59e6c7a517e37458e1a6c3daf commit, or I miss something? cc @AlexDaniel

melezhik commented 4 years ago

@AlexDaniel , try it out!

curl -d os=debian -d rakudo_version=add2ec0d491218e59e6c7a517e37458e1a6c3daf http://repo.westus.cloudapp.azure.com/rakudist/api/run/ADT

Limitation of the first version:

I'll make further improvements in the next versions of API.

Please try and let me know how it goes!

melezhik commented 4 years ago

Examples:

AlexDaniel commented 4 years ago

Hold on I'll give you a way to get the full sha. It will be available in response headers. It's a bit unorthodox to pass extra data in headers, but it should work fine. You may start figuring out how to get the header value using wget or whatever :)

AlexDaniel commented 4 years ago

Done. If you look at the headers, you'll see something like this:

image

For .zstd archives that string is part of the filename, but for older .lrz archives it will differ (because one archive can contain more than one build).

melezhik commented 4 years ago

Thanks. To parse a header is not a big deal, till i get it done, just provide a full shai as I mentioned.

On Jan 18, 2020 8:09 PM, "Aleks-Daniel Jakimenko-Aleksejev" < notifications@github.com> wrote:

Done. If you look at the headers, you'll see something like this:

[image: image] https://user-images.githubusercontent.com/5507503/72673318-4285c580-3a71-11ea-87a8-ef50f2e81b54.png

For .zstd archives that string is part of the filename, but for older .lrz archives it will differ (because one archive can contain more than one build).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/melezhik/RakuDist/issues/1?email_source=notifications&email_token=AAHRHSKNICQEHYA3AICWRP3Q6OY33A5CNFSM4KIO3GQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJKGY4Q#issuecomment-575958130, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHRHSLB4ULP77KDI74OER3Q6OY33ANCNFSM4KIO3GQQ .

melezhik commented 4 years ago

@AlexDaniel have you had a chance to check an API? Does it work for you?

AlexDaniel commented 4 years ago

@melezhik when running it from a script, how can I wait for the process to finish?

For example, if I run this:

curl -d os=debian -d rakudo_version=add2ec0d491218e59e6c7a517e37458e1a6c3daf http://repo.westus.cloudapp.azure.com/rakudist/api/run/ADT

It responds:

0 build(s) running
Run default test for ADT on debian rakudo version add2ec0d491218e59e6c7a517e37458e1a6c3daf
test launched, it takes a while
see report at http://repo.westus.cloudapp.azure.com/rakudist/reports/ADT/debian/1579467800.txt

Which is fine, I can get that last url. But how would I know when it finishes? Also, can I get something like an exit code so that I can know if it failed or not without parsing the output?

melezhik commented 4 years ago

The request statues are not yet implemented. I think I will add this feature very soon. So far it would be interesting to test the API with different modules/version to clear all other possible issues with it.

AlexDaniel commented 4 years ago

@melezhik well, if we want to get people using it conveniently through committable6/bisectable6, I need a way to “get the output”. Without it I'm not entirely sure what I can do. Ideally I just need the bash: Install … part (without timestamps), but I can filter that out myself easily. The biggest problem is figuring out when the result is ready. Is there any quick way we can get it?

AlexDaniel commented 4 years ago

What about printing some easily parseable string at the end? Bonus points if it includes the exit code too. Then I'll just query the url every second.

melezhik commented 4 years ago

@AleksDaniel , I see. I could suggest synchronous responses as an alternative, though the average time for those ones could be 1-3 munutes. Is it ok? I would return report data in body and exit code as a http header immediately in one request/response.

This approach will solve an immediate need of bisectable or other clients, though in a long run current "asynchronous" RakuDistApi model is more easy to scale and handle high load.

What do you think to choose? I am still ok with what you've suggested before and returning exit code as well as a specific log marker indicating that a test has finished in the end of the report is also possible.

On Jan 19, 2020 3:36 PM, "Aleks-Daniel Jakimenko-Aleksejev" < notifications@github.com> wrote:

What about printing some easily parseable string at the end? Bonus points if it includes the exit code too. Then I'll just query the url every second.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/melezhik/RakuDist/issues/1?email_source=notifications&email_token=AAHRHSKGTIZFMHO2HX7G4YTQ6TBUNA5CNFSM4KIO3GQ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJK5G3I#issuecomment-576050029, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHRHSMEO4IBVOHZH33NY6LQ6TBUNANCNFSM4KIO3GQQ .

AlexDaniel commented 4 years ago

I could suggest synchronous responses as an alternative, though the average time for those ones could be 1-3 minutes. Is it ok?

Sounds perfect.

What do you think to choose?

The first option is just a little bit easier to implement for me, but otherwise I don't care.

melezhik commented 4 years ago

@AlexDaniel implemented, you should add new parameter to a request:

Http header X-RakuDist-ExitCode will contain an exit code.

So a request will be:

curl -d os=$os -d rakudo_version=$version -d sync_mode=on http://repo.westus.cloudapp.azure.com/rakudist/api/run/$module

Example:

[melezhik@localhost ~]$ curl -d os=debian -d rakudo_version=add2ec0d491218e59e6c7a517e37458e1a6c3daf -d sync_mode=on http://repo.westus.cloudapp.azure.com/rakudist/api/run/ADT -D  - 
HTTP/1.1 200 OK
Server: nginx/1.12.2
Date: Mon, 20 Jan 2020 00:35:41 GMT
Content-Type: text/html;charset=UTF-8
Content-Length: 7896
Connection: keep-alive
X-RakuDist-ExitCode: 256

0 build(s) running
Run default test for ADT on debian. Rakudo version add2ec0d491218e59e6c7a517e37458e1a6c3daf
You can replay this test using RakuDist - https://github.com/melezhik/RakuDist
===
00:34:44 01/20/2020 [repository] index updated from http://repo.westus.cloudapp.azure.com/api/v1/index
Start Rakudo install, version add2ec0d491218e59e6c7a517e37458e1a6c3daf
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] trying to install wget ...
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] installer - apt-get
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] Package: wget
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] Version: 1.20.1-1.1
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] Status: install ok installed
[task check] stdout match <Status: install ok installed> True
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] trying to install zstd ...
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] installer - apt-get
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] Package: zstd
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] Version: 1.3.8+dfsg-3
00:34:47 01/20/2020 [install package(s): ("wget", "zstd")] Status: install ok installed
[task check] stdout match <Status: install ok installed> True
00:34:51 01/20/2020 [create directory /data/whateverable/] directory path: /data/whateverable/
00:34:51 01/20/2020 [create directory /data/whateverable/] directory owner: <root>
00:34:51 01/20/2020 [create directory /data/whateverable/] directory group: <root>
00:34:51 01/20/2020 [create directory /data/whateverable/] directory access rights: drwxr-xr-x
00:34:54 01/20/2020 [bash: unpack add2ec0d491218e59e6c7a517e37458e1a6c3daf] <empty stdout>
00:34:58 01/20/2020 [create user cpan-adt-debian] Don't change user home as managehome set to 'no' OR homedir not set
00:34:58 01/20/2020 [create user cpan-adt-debian] user cpan-adt-debian - nothing changed
00:34:58 01/20/2020 [create user cpan-adt-debian] uid=1005(cpan-adt-debian) gid=1005(cpan-adt-debian) groups=1005(cpan-adt-debian)
00:35:02 01/20/2020 [create directory /home/cpan-adt-debian/zef] directory path: /home/cpan-adt-debian/zef
00:35:02 01/20/2020 [create directory /home/cpan-adt-debian/zef] directory owner: <cpan-adt-debian>
00:35:02 01/20/2020 [create directory /home/cpan-adt-debian/zef] directory group: <root>
00:35:02 01/20/2020 [create directory /home/cpan-adt-debian/zef] directory access rights: drwxr-xr-x
[task check] stdout match <owner: <cpan-adt-debian>> True
00:35:05 01/20/2020 [bash: git checkout https://github.com/ugexe/zef.git] /home/cpan-adt-debian/zef
00:35:06 01/20/2020 [bash: git checkout https://github.com/ugexe/zef.git] Already up to date.
Start Raku module test, module ADT
00:35:08 01/20/2020 [delete directory /home/cpan-adt-debian/.perl6] directory-delete-ok
[task check] stdout match <directory-delete-ok> True
00:35:11 01/20/2020 [create directory /home/cpan-adt-debian/.perl6] directory path: /home/cpan-adt-debian/.perl6
00:35:11 01/20/2020 [create directory /home/cpan-adt-debian/.perl6] directory owner: <cpan-adt-debian>
00:35:11 01/20/2020 [create directory /home/cpan-adt-debian/.perl6] directory group: <cpan-adt-debian>
00:35:12 01/20/2020 [create directory /home/cpan-adt-debian/.perl6] directory access rights: drwxr-xr-x
[task check] stdout match <owner: <cpan-adt-debian>> True
[task check] stdout match <group: <cpan-adt-debian>> True
00:35:15 01/20/2020 [delete directory /home/cpan-adt-debian/.zef] directory-delete-ok
[task check] stdout match <directory-delete-ok> True
00:35:19 01/20/2020 [create directory /home/cpan-adt-debian/.zef] directory path: /home/cpan-adt-debian/.zef
00:35:19 01/20/2020 [create directory /home/cpan-adt-debian/.zef] directory owner: <cpan-adt-debian>
00:35:19 01/20/2020 [create directory /home/cpan-adt-debian/.zef] directory group: <cpan-adt-debian>
00:35:19 01/20/2020 [create directory /home/cpan-adt-debian/.zef] directory access rights: drwxr-xr-x
[task check] stdout match <owner: <cpan-adt-debian>> True
[task check] stdout match <group: <cpan-adt-debian>> True
00:35:23 01/20/2020 [bash: rakudo version] This is Rakudo version 2019.11-377-gadd2ec0d4 built on MoarVM version 2019.11-109-ge94893f89
00:35:23 01/20/2020 [bash: rakudo version] implementing Perl 6.d.
00:35:26 01/20/2020 [bash: Install ADT] ===> Searching for: ADT
00:35:26 01/20/2020 [bash: Install ADT] stderr: ===> Updating cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
00:35:41 01/20/2020 [bash: Install ADT] ===> Testing: ADT:ver<0.5>:auth<Timo Paulssen>
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # Failed test 'evaling a construction gists out exactly the same again.'
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # at t/01-tree.t line 13
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # expected: 'Tree.new-branch(left => Tree.new-branch(left => Tree.new-leaf(storage => 1), right => Tree.new-leaf(storage => 2)), right => Tree.new-leaf(storage => 3))'
00:35:41 01/20/2020 [bash: Install ADT] [ADT] #      got: 'Tree.new-branch(left => Tree.new(branch => Branch.new(left => Tree.new(branch => Any, leaf => Leaf.new(storage => 1)), right => Tree.new(branch => Any, leaf => Leaf.new(storage => 2))), leaf => Any), right => Tree.new(branch => Any, leaf => Leaf.new(storage => 3)))'
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # Failed test 'evaling a construction perls out exactly the same again.'
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # at t/01-tree.t line 14
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # expected: 'Tree.new-branch(left => Tree.new-branch(left => Tree.new-leaf(storage => 1), right => Tree.new-leaf(storage => 2)), right => Tree.new-leaf(storage => 3))'
00:35:41 01/20/2020 [bash: Install ADT] [ADT] #      got: 'Tree.new-branch(left => Tree.new(branch => Branch.new(left => Tree.new(branch => Any, leaf => Leaf.new(storage => 1)), right => Tree.new(branch => Any, leaf => Leaf.new(storage => 2))), leaf => Any), right => Tree.new(branch => Any, leaf => Leaf.new(storage => 3)))'
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # Failed test 'positional args for constructors work, too'
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # at t/01-tree.t line 22
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # expected: 'Tree.new-branch(left => Tree.new-branch(left => Tree.new-leaf(storage => 1), right => Tree.new-leaf(storage => 2)), right => Tree.new-leaf(storage => 3))'
00:35:41 01/20/2020 [bash: Install ADT] [ADT] #      got: 'Tree.new-branch(left => Tree.new(branch => Branch.new(left => Tree.new(branch => Any, leaf => Leaf.new(storage => 1)), right => Tree.new(branch => Any, leaf => Leaf.new(storage => 2))), leaf => Any), right => Tree.new(branch => Any, leaf => Leaf.new(storage => 3)))'
00:35:41 01/20/2020 [bash: Install ADT] [ADT] # You failed 3 tests of 7
00:35:41 01/20/2020 [bash: Install ADT] ===> Testing [FAIL]: ADT:ver<0.5>:auth<Timo Paulssen>
00:35:41 01/20/2020 [bash: Install ADT] stderr: ===> Updating p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated p6c mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/p6c1.json
===> Updated cpan mirror: https://raw.githubusercontent.com/ugexe/Perl6-ecosystems/master/cpan1.json
Aborting due to test failure: ADT:ver<0.5>:auth<Timo Paulssen> (use --force-test to override)
00:35:41 01/20/2020 [bash: Install ADT] task exit status: 1
00:35:41 01/20/2020 [bash: Install ADT] task bash: Install ADT FAILED
The spawned command 'docker exec -i debian-rakudist sh /root/.sparrowdo/sparrowrun.sh' exited unsuccessfully (exit code: 1)
  in block <unit> at /opt/rakudo-pkg/share/perl6/site/resources/375DCCC7CEEDC0A696B38FBD13B6A1EBB162B393 line 17
  in sub MAIN at /home/scheck/.perl6/bin/sparrowdo line 3
  in block <unit> at /home/scheck/.perl6/bin/sparrowdo line 1
melezhik commented 4 years ago

And a passed test:

[melezhik@localhost ~]$ curl -d os=debian -d rakudo_version=40b13322c503808235d9fec782d3767eb8edb899 -d sync_mode=on http://repo.westus.cloudapp.azure.com/rakudist/api/run/ADT -D  - 
HTTP/1.1 200 OK
Server: nginx/1.12.2
Date: Mon, 20 Jan 2020 00:41:34 GMT
Content-Type: text/html;charset=UTF-8
Content-Length: 4485
Connection: keep-alive
X-RakuDist-ExitCode: 0

0 build(s) running
Run default test for ADT on debian. Rakudo version 40b13322c503808235d9fec782d3767eb8edb899
You can replay this test using RakuDist - https://github.com/melezhik/RakuDist
===
00:40:28 01/20/2020 [repository] index updated from http://repo.westus.cloudapp.azure.com/api/v1/index
Start Rakudo install, version 40b13322c503808235d9fec782d3767eb8edb899
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] trying to install wget ...
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] installer - apt-get
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] Package: wget
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] Version: 1.20.1-1.1
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] Status: install ok installed
[task check] stdout match <Status: install ok installed> True
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] trying to install zstd ...
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] installer - apt-get
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] Package: zstd
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] Version: 1.3.8+dfsg-3
00:40:33 01/20/2020 [install package(s): ("wget", "zstd")] Status: install ok installed
[task check] stdout match <Status: install ok installed> True
00:40:39 01/20/2020 [create directory /data/whateverable/] directory path: /data/whateverable/
00:40:39 01/20/2020 [create directory /data/whateverable/] directory owner: <root>
00:40:39 01/20/2020 [create directory /data/whateverable/] directory group: <root>
00:40:39 01/20/2020 [create directory /data/whateverable/] directory access rights: drwxr-xr-x
00:40:44 01/20/2020 [bash: unpack 40b13322c503808235d9fec782d3767eb8edb899] <empty stdout>
00:40:51 01/20/2020 [create user cpan-adt-debian] Don't change user home as managehome set to 'no' OR homedir not set
00:40:51 01/20/2020 [create user cpan-adt-debian] user cpan-adt-debian - nothing changed
00:40:51 01/20/2020 [create user cpan-adt-debian] uid=1005(cpan-adt-debian) gid=1005(cpan-adt-debian) groups=1005(cpan-adt-debian)
00:40:56 01/20/2020 [create directory /home/cpan-adt-debian/zef] directory path: /home/cpan-adt-debian/zef
00:40:56 01/20/2020 [create directory /home/cpan-adt-debian/zef] directory owner: <cpan-adt-debian>
00:40:56 01/20/2020 [create directory /home/cpan-adt-debian/zef] directory group: <root>
00:40:56 01/20/2020 [create directory /home/cpan-adt-debian/zef] directory access rights: drwxr-xr-x
[task check] stdout match <owner: <cpan-adt-debian>> True
00:41:02 01/20/2020 [bash: git checkout https://github.com/ugexe/zef.git] /home/cpan-adt-debian/zef
00:41:02 01/20/2020 [bash: git checkout https://github.com/ugexe/zef.git] Already up to date.
Start Raku module test, module ADT
00:41:06 01/20/2020 [delete directory /home/cpan-adt-debian/.perl6] directory-delete-ok
[task check] stdout match <directory-delete-ok> True
00:41:11 01/20/2020 [create directory /home/cpan-adt-debian/.perl6] directory path: /home/cpan-adt-debian/.perl6
00:41:11 01/20/2020 [create directory /home/cpan-adt-debian/.perl6] directory owner: <cpan-adt-debian>
00:41:11 01/20/2020 [create directory /home/cpan-adt-debian/.perl6] directory group: <cpan-adt-debian>
00:41:11 01/20/2020 [create directory /home/cpan-adt-debian/.perl6] directory access rights: drwxr-xr-x
[task check] stdout match <owner: <cpan-adt-debian>> True
[task check] stdout match <group: <cpan-adt-debian>> True
00:41:18 01/20/2020 [delete directory /home/cpan-adt-debian/.zef] directory-delete-ok
[task check] stdout match <directory-delete-ok> True
00:41:23 01/20/2020 [create directory /home/cpan-adt-debian/.zef] directory path: /home/cpan-adt-debian/.zef
00:41:23 01/20/2020 [create directory /home/cpan-adt-debian/.zef] directory owner: <cpan-adt-debian>
00:41:23 01/20/2020 [create directory /home/cpan-adt-debian/.zef] directory group: <cpan-adt-debian>
00:41:23 01/20/2020 [create directory /home/cpan-adt-debian/.zef] directory access rights: drwxr-xr-x
[task check] stdout match <owner: <cpan-adt-debian>> True
[task check] stdout match <group: <cpan-adt-debian>> True
00:41:29 01/20/2020 [bash: rakudo version] This is Rakudo version 2019.07.1 built on MoarVM version 2019.07.1
00:41:29 01/20/2020 [bash: rakudo version] implementing Perl 6.d.
00:41:34 01/20/2020 [bash: Install ADT] stderr: All candidates are currently installed
00:41:34 01/20/2020 [bash: Install ADT] No reason to proceed. Use --force-install to continue anyway
AlexDaniel commented 4 years ago

Looks good to me! I'll try plug this into committable as soon as possible.

melezhik commented 4 years ago

@AlexDaniel please hold on with testing, Afters some testing I revealed too many 499 errors I am sorting things out ..

melezhik commented 4 years ago

@AlexDaniel I've fixed the issue, by changing RakuDistAPI backend to Mojolicious prefox server which is the best to handle long running requests, you can start using API now. Please let me know if you had any issues. Please pay attention - it's relatively small VM, with 5 workers behind Mojolicous, so try not to overload it by requests ((=:

melezhik commented 4 years ago

Hi @AlexDaniel any news here?

melezhik commented 4 years ago

@AlexDaniel I've implemented job statuses, so there is no need in synchronous requests:

https://github.com/melezhik/RakuDist/blob/master/README.md#run-tests-via-api

Let's use asynchronous approach as it more reliable:

token=$(curl -s -d os=debian http://repo.westus.cloudapp.azure.com/rakudist/api/run/Kind)
while true; do
  status=$(curl -s -d token=$token http://repo.westus.cloudapp.azure.com/rakudist/job/status)
  if test $status != "running"; then
    break
  fi
done
echo "test: $status"
curl -s -d token=$token http://repo.westus.cloudapp.azure.com/rakudist/job/report
melezhik commented 4 years ago

@AlexDaniel . correct example:

token=$(curl -s -d os=debian http://repo.westus.cloudapp.azure.com/rakudist/api/run/Kind)
echo $token
while true; do
  status=$(curl -s -d token=$token http://repo.westus.cloudapp.azure.com/rakudist/api/job/status)
  sleep 5
  echo $status
  if [ $status != "running" ]; then
    break
  fi
done
echo "test: $status"
curl -L -s -d token=$token http://repo.westus.cloudapp.azure.com/rakudist/api/job/report
AlexDaniel commented 4 years ago

Hi! Sorry for some inactivity, I'm trying to recover from the surgery and also to prepare for something I have to do IRL. As for the status API, I'm not sure if I like it. If I have to sleep between the checks it means that this latency adds up for anything that needs to be done sequentially (e.g. bisection). So in the end you'll have a few seconds wasted just on waiting for the status. It's not much, but it counts. I don't know what'd be the modern way to do it, like websockets maybe? In the past long-polling worked fine, but I guess it's not fancy enough :)

melezhik commented 4 years ago

Hi! I don't know, maybe I don't understand your context, but current API server capacities are still not enough to handle more then 4 concurrent jobs running ... So in this case imho it does not really matter which model we choose (straightforward polling or whatever), in any case it won't increase productivity as a bottleneck the server itself. But let's start testing it and we will see real problems if any ...

AlexDaniel commented 4 years ago

but current API server capacities are still not enough to handle more then 4 concurrent jobs running

That's the point. If you're running them sequentially, then just the delay for checking the status can add up (e.g. if you're testing 10 revisions and your delay is 2s then you waste around 10s just on waiting). Yes it's not a lot compared to the total run time, but in my view it's significant. Of course I can spam the API every 200ms or so, but I hope you see my point. Anyway, that's just food for thought.

melezhik commented 4 years ago

That's the point. If you're running them sequentially,

Why not to run them ( http requests through curl ) in parallel? Say 4 jobs in parallel?

AlexDaniel commented 4 years ago

@melezhik sure. But as I said, not all jobs are parallelizable. Bisection is not.

melezhik commented 4 years ago

I am sorry, why it's not possible to do stuff in parallel in Bisection? For example you can run 3 asynchronous request, that don't block script:

token1=$(curl -s -d os=debian http://repo.westus.cloudapp.azure.com/rakudist/api/run/Kind)
token2=$(curl -s -d os=debian http://repo.westus.cloudapp.azure.com/rakudist/api/run/Tomty)
token3=$(curl -s -d os=debian http://repo.westus.cloudapp.azure.com/rakudist/api/run/App::GTD)

and then wait for them ( whichever script finishes first ... ):

while true; do
  status1=$(curl -s -d token=$token1 http://repo.westus.cloudapp.azure.com/rakudist/api/job/status)
  if [ $status1 != "running" ]; then
    echo "first is done"
    report1=$(curl -s -d token=$token1 http://repo.westus.cloudapp.azure.com/rakudist/api/job/report)
  fi
  status2=$(curl -s -d token=$token2 http://repo.westus.cloudapp.azure.com/rakudist/api/job/status)
  if [ $status2 != "running" ]; then
    echo "second is done"
    report2=$(curl -s -d token=$token2 http://repo.westus.cloudapp.azure.com/rakudist/api/job/report)
  fi
  status3=$(curl -s -d token=$token3 http://repo.westus.cloudapp.azure.com/rakudist/api/job/status)
  if [ $status3 != "running" ]; then
    echo "third is is done"
    report3=$(curl -s -d token=$token3 http://repo.westus.cloudapp.azure.com/rakudist/api/job/report)

  fi
end
AlexDaniel commented 4 years ago

@melezhik because when you're bisecting rakudo for a single module you need to test ≈12 revisions or so. While you're testing the first one you can't know which second revision you'll need because it depends on the result of the first test. Essentially it'll be one of two possibilities, and you can do a bit of lookahead, but every lookahead step requires twice as many revisions than the previous one, which negates the whole point of bisection. And even if you do it just 1 step ahead, it's still wasteful. In the end whateverable only tests stuff sequentially, so that's ≈12 steps unparallelized, and with a bit of lookahead this could maybe be reduced to 6 but not more.

Again, the amount of time wasted on polling timeouts is negligible compared to the total run time, but I don't like waiting extra seconds and I think the API itself can be better, but it's no biggie.

melezhik commented 4 years ago

but I don't like waiting extra seconds and I think the API itself can be better, but it's no biggie.

sure 😄

so that's ≈12 steps unparallelized

I see. I thought on paralyzing on module basis, not commits, but it's probable impossible with Bisecting?

AlexDaniel commented 4 years ago

I see. I thought on paralyzing on module basis, not commits, but it's probable impossible with Bisecting?

Right. Blin does a very good job at bisecting the whole ecosystem, and I don't think RakuDist should or even can compete with that. I see RakuDist and its API as a great way for authors or devs to test or bisect specific modules.

AlexDaniel commented 4 years ago

I'm a bit confused. Besides the cool API, is there any convenient way for a regular user to use it? It used to respond with a URL where it was possible to follow the progress, but not anymore. Is there any tokenless way to use it currently?

AlexDaniel commented 4 years ago

Ah, I found this: http://repo.westus.cloudapp.azure.com/rakudist/reports/Array%3A%3AShaped%3A%3AConsole/debian/ That said, it doesn't seem to be working…

melezhik commented 4 years ago

Hi @AlexDaniel , should work now. Please try again. Please pay attention that only Debian docker instance is available right now.

PS the reason of the error is I forgot to sudo chmod 666 /var/run/docker.sock after the VM reboot

melezhik commented 4 years ago

Hi @AlexDaniel , works for me now, please let me know if you need any help ...

http://repo.westus.cloudapp.azure.com/rakudist/reports/Array%3A%3AShaped%3A%3AConsole/debian/1585312228.txt