search

EcoJulia / Microbiome.jl

For analysis of microbiome and microbial community data
Other
46 stars 10 forks source link

Register v0.4.1 #30

Closed kescobo closed 5 years ago

kescobo commented 5 years ago

@JuliaRegistrator register()

TransGirlCodes commented 5 years ago

This might not work, I'm working on getting a BioJulia registry set up for us here: https://github.com/BioJulia/BioJuliaRegistry, and I am experimenting with the Registrator bot myself locally and on Heroku to get it running for us.

kescobo commented 5 years ago

Noted. Do you know why it's not working?

TransGirlCodes commented 5 years ago

JuliaRegistrator hasn't been turned on for any BioJulia packages. Because I intend for us to have a BioJuliaRegistrator, which does the same job, but that registers BioJulia packages on a BioJulia registry instead of General. Did you want to test out the General registry?

kescobo commented 5 years ago

No, I just wanted to make a new release, I didn't realize we weren't going to be on General. Was that discussed in an issue somewhere?

TransGirlCodes commented 5 years ago

Was that discussed in an issue somewhere?

Admittedly no, it was an idea I came up with when:

I know some people would rather everything be in one monolithic registry. But to me that argument seems to rest on "I don't want to install multiple registries". To me that seems to defy the point of some of Pkg3: One of the upgrades from Pkg2 is it's registry support, and people from JuliaComputing when mentioning registry support have often mentioned that open source orgs would be able to have their own registries, which sounds like they expect it to happen (and it implies they're all for it).

Field-specific registries make sense to me in a Pkg3 world. We would have more control of when our stuff gets merged into a BioJulia registry. METADATA got a lot better eventually, but sometimes we'd be waiting weeks for a release to go through in the worse cases.

We'd also then have a mechanism through which we can endorse packages built by others using BioJulia, or that are compatible with BioJulia: they can be made available through our registry and listed on our website (which I'm in the middle of revamping with Hugo).

I am happy to discuss the decision with people, but I would just say let's try it and see how it goes, we lose nothing if we change our minds, because AFAIK, a package can be listed in multiple registries, so it's not like people have to give up General if they really hate the prospect.

kescobo commented 5 years ago

To me that seems to defy the point of some of Pkg3: One of the upgrades from Pkg2 is it's registry support, and people from JuliaComputing when mentioning registry support have often mentioned that open source orgs would be able to have their own registries, which sounds like they expect it to happen (and it implies they're all for it).

I had this thought originally too, but there was a thread where someone (I think Stephan or Kristoffer) seemed to indicate that everything should go in General.

I am happy to discuss the decision with people, but I would just say let's try it and see how it goes, we lose nothing if we change our minds, because AFAIK, a package can be listed in multiple registries, so it's not like people have to give up General if they really hate the prospect.

A corollary to this is that we could put everything in General and switch to BioRegistry (or whatever) at a later time. In any case, I think it would be worth discussing on discourse or in an issue. I think I agree with you so long as:

1) We're able to get it up and running without much delay 2) Other people in the org know how to maintain it other than you 2b) it stays maintained

You seem to have a lot of irons in a lot of fires, which is awesome, but I'd hate for you to add this to your plate (excuse the mixed metaphors) and have other projects complaining to you when things go more slowly than they'd like.

TransGirlCodes commented 5 years ago

I had this thought originally too, but there was a thread where someone (I think Stephan or Kristoffer) seemed to indicate that everything should go in General.

Stefan offered to have JuliaComputing host an instance of the Registrator bot for BioJulia. It wouldn't surprise me if even within JuliaComputing, there was variance in the personal opinions of the members on this.

A corollary to this is that we could put everything in General and switch to BioRegistry (or whatever) at a later time.

For the packages already in General, this is exactly what we would be doing anyway. But there could be a phasing period if you like. New packages like Pseudoseq and the assembly and graph package could be registered using the new registry, and the older ones moved over gradually when everyone is happy with the new registering method. But for me the speed at which things should happen is one area open for discussion - the move from Bio.jl to the other package organisation was a slower one, and people still seem to be caught out by it sometimes.

1). We're able to get it up and running without much delay. 2) Other people in the org know how to maintain it other than you. 2b) It stays maintained.

You seem to have a lot of irons in a lot of fires, which is awesome, but I'd hate for you to add this to your plate (excuse the mixed metaphors) and have other projects complaining to you when things go more slowly than they'd like.

In a work environment where academics are constantly told "more do more" I really appreciate the concern. With regards to points 1, 2, and 2b, I've already been working on getting Registrator working on Heroku, and there is currently an instance of it running: BioJulia has an app called "BioJulia Registrar", that you can test out by installing it on Microbiome.jl. You can see the issue I've been testing it on here: https://github.com/BioJulia/FASTX.jl/issues/2 and the resulting PR here: https://github.com/BioJulia/BioJuliaRegistry/pull/1

Hosting these instances on Heroku's free tier (appropriate I think for the frequency it will be used) is more reliable that having something running on a localhost machine (or a RPi in my home or office!!!), it also means that other BioJulia members that want to take a role assisting with maintaining this would be able to start and stop the app and check on it and so forth, as we can make a team on Heroku.

Finally I've created a bot user (currently called BioJuliaRegistrar, but I'm thinking of naming it Helix), which will be used for both the registrar bot and other bots we may set up in the future - meaning the system will not depend on anyone's personal user acct. In the future I'd like Helix to be responsible for other automated things too, like checking repos have up to date Codes of Conduct files and so on, relieving the burden of any maintainer.

kescobo commented 5 years ago

@BenJWard Not sure how I missed this for 3 weeks - being a new parent is rough!

How do I install the Registrar App? Couldn't find any docs on the new registry.

@bioreg register()

kescobo commented 5 years ago

Might be good to put some docs for how to add packages that are found in the biojulia registry too, since I'm guessing this is still a bit niche.

TransGirlCodes commented 5 years ago

@BenJWard Not sure how I missed this for 3 weeks - being a new parent is rough!

How do I install the Registrar App? Couldn't find any docs on the new registry.

@bioreg register()

I can imagine! My supervisor was a new parent recently and he's being stretched to the limit.

The app is public and can be installed by anyone (which is how I got Pseudoseq on there), although admittedly the link to install it is a bit obscure. I've gone ahead and turned the BioJulia registrar on for all BioJulia packages. So if you try the bioreg register comment again, it should work this time.

I'm in the process of testing an alternative way of registering packages, by visiting a web UI (like this https://giant.gfycat.com/ImpracticalFrightenedAsiandamselfly.webm) which people might find easier. I just need to work up a PR to Registrator.jl so as the web UI can accept a list of extra registries in which to find dependencies (General) and then we're laughing.

TransGirlCodes commented 5 years ago

Might be good to put some docs for how to add packages that are found in the biojulia registry too, since I'm guessing this is still a bit niche.

Docs for this will be coming very soon. I've been working on a new page for biojulia.net, built using hugo-academic, which includes a "getting-started" section, which explains to the new user (after they have been pointed to do the first thing - install julia), the out of the box, julia's PM only watches "General", and shows how to add BioJuliaRegistry with registry add in Pkg mode.

kescobo commented 5 years ago

@bioreg register()

BioJuliaBot commented 5 years ago

Registration pull request created: BioJulia/BioJuliaRegistry/12

Optionally, you can create a tag on this repository for the above registration.

git tag -a v0.4.1 -m "my version 0.4.1" 0cf5eb5c0a230eb060a894d740e07ce49b819a8d
git push --tags
BioJuliaBot commented 5 years ago

An unexpected error occured during registration.

BioJuliaBot commented 5 years ago

An unexpected error occured during registration.

kescobo commented 5 years ago

@BenJWard Thoughts? Do I just need to approve it?

TransGirlCodes commented 5 years ago

That looks like it worked, not sure what the errors are about. The approve mechanism for the bot AFAIK is for when someone triggers the bot to register a PR, because the PR needs to be registered, and then merged. Triggering register from this issue should just have it register whatever the state of master of Microbiome.jl is. In which case the only thing left to do is to merge the PR over at the registry, and for you to tag or a do a github release (which does not affect the registry at all like it did Attobot, but its just more for human book-keeping now).

TransGirlCodes commented 5 years ago 
N82106:registrator bward$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.1.0 (2019-01-21)
 _/ |\__'_|_|_|\__'_|  |  
|__/                   |

(v1.1) pkg> add Microbiome
  Updating registry at `~/.julia/registries/BioJuliaRegistry`
  Updating git-repo `https://github.com/BioJulia/BioJuliaRegistry.git`
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
 Resolving package versions...
 Installed CMakeWrapper ────────── v0.2.3
 Installed EcoBase ─────────────── v0.0.7
 Installed RandomBooleanMatrices ─ v0.1.1
 Installed PooledArrays ────────── v0.5.1
 Installed Microbiome ──────────── v0.4.1
 Installed RandomNumbers ───────── v1.2.0
 Installed NearestNeighbors ────── v0.4.3
 Installed SpatialEcology ──────── v0.7.0
 Installed Mustache ────────────── v0.5.12
 Installed DataFrames ──────────── v0.18.2
 Installed DataFramesMeta ──────── v0.4.1
 Installed Clustering ──────────── v0.13.1
  Updating `~/.julia/environments/v1.1/Project.toml`
  [3bd8f0ae] + Microbiome v0.4.1
  Updating `~/.julia/environments/v1.1/Manifest.toml`
  [d5fb7624] ↑ CMakeWrapper v0.2.2 ⇒ v0.2.3
  [324d7699] + CategoricalArrays v0.5.2
  [aaaa29a8] + Clustering v0.13.1
  [a93c6f00] + DataFrames v0.18.2
  [1313f7d8] + DataFramesMeta v0.4.1
  [b4f34e82] + Distances v0.8.0
  [a58aae7d] + EcoBase v0.0.7
  [3bd8f0ae] + Microbiome v0.4.1
  [ffc61752] ↑ Mustache v0.5.11 ⇒ v0.5.12
  [b8a86587] + NearestNeighbors v0.4.3
  [2dfb63ee] + PooledArrays v0.5.1
  [9ae346a0] + RandomBooleanMatrices v0.1.1
  [e6cf234a] + RandomNumbers v1.2.0
  [348f2d5d] + SpatialEcology v0.7.0
  [9fa8497b] + Future 

julia> using Microbiome
[ Info: Precompiling Microbiome [3bd8f0ae-a0f2-5238-a5af-e1b399a4940c]

julia>

Looks like it works to me!

TransGirlCodes commented 5 years ago

Perhaps it got confused with some comment like when I quoted you calling the register command the first time!

kescobo commented 5 years ago

Oh neat. Cool, thanks!