search

raaz-crypto / raaz

Cryptographic library for Haskell
Apache License 2.0
68 stars 24 forks source link

Uploading candidate release onto Debian experimental #341

Closed piyush-kurur closed 7 years ago

piyush-kurur commented 7 years ago

Debian build bots build across architectures and even across platforms Hurd, kFreeBSD etc. A successful build is an automatic guarantee of quality of the package. It would therefore be good to upload candidate packages on to debian experimental.

Here is the link to the candidate release

https://hackage.haskell.org/package/raaz-0.2.0/candidate/raaz-0.2.0.tar.gz

piyush-kurur commented 7 years ago

@spwhitton Can you upload this https://hackage.haskell.org/package/raaz-0.2.0/candidate/raaz-0.2.0.tar.gz and let me know if things went fine. If it is okey, I will do an actual release.

spwhitton commented 7 years ago

On Wed, Aug 09 2017, Piyush P. Kurur wrote:

@spwhitton Can you upload this https://hackage.haskell.org/package/raaz-0.2.0/candidate/raaz-0.2.0.tar.gz and let me know if things went fine. If it is okey, I will do an actual release.

Working on this now.

In future, please make a release with a version number like this: 0.2.0~rc1. This sorts before 0.2.0 in Debian's toolchain. Don't worry in this case.

Results will be available from https://buildd.debian.org as usual.

-- Sean Whitton

spwhitton commented 7 years ago

Hello,

On Thu, Aug 10 2017, Sean Whitton wrote:

Working on this now.

Unfortunately it looks like our tooling doesn't work well with this "candidate" machinery you're using on hackage.

I don't want to work around that tooling as it will make life harder for the other people on the Debian Haskell team.

-- Sean Whitton

spwhitton commented 7 years ago

... sorry, my e-mail was cut off.

Can you make an ordinary release on hackage but with the ~rc1 suffix, please? Then I can upload to experimental using our ordinary tools.

piyush-kurur commented 7 years ago

I am not sure if this is possible on hackage. If I release 0.2.0.0 something then the next release can only be 0.2.0.1 (which is not bad). If the debian tooling is not there, I am fine by releasing 0.2.0.0 and then patching it to 0.2.0.1 in case of bugs. I just wanted to know if there is a better way out.

So feel free to close this ticket with a wontfix.

piyush-kurur commented 7 years ago

@hvr do you have any comments on this since you are familiar with both the debian side and the Hackage side.

hvr commented 7 years ago

@piyush-kurur I'm not sure what @spwhitton means. There is no such thing as "~rc1" suffixes on Hackage. "ordinary releases" are not release candidates, they are proper releases intended to be final. "Hackage release candidates" are the closest thing to "release candidates" we have. And as you pointed out correctly, once you publish "0.2.0.0" to the primary pkg index, that version number has been used up for good. This also means, that once you publish e.g. 0.2.0.0, the PVP starts applying; i.e. if you realise you need to make a backward incompatible change relative to 0.2.0.0, then you can't use 0.2.0.1, but you have to go for 0.3.0.0.

piyush-kurur commented 7 years ago

@hvr thanks for the clarification so it looks like I will have to do a 0.2.0.0 release and then see what debian build bots say.

The reason I asked this was there was an endian related compilation bug in the previous release 0.1 which got exposed thanks to the debian build bot. I was hoping that if debian buld bots can work with the release candidate then that would be good. I am closing this ticket because this does not seem possible.

spwhitton commented 7 years ago

Hello @piyush-kurur. I'm sorry that Debian couldn't be more helpful in this case. Just for the record, it's not our central infrastructure that is the problem -- it's the git infrastructure used by the Haskell team.

If you want me to try the build on a particular architecture, I can manually do that without uploading to experimental, and you don't need to make a release. Just let me know.

piyush-kurur commented 7 years ago

@spwhitton it is okey. No problems. Do not burden yourself.

spwhitton commented 7 years ago

@piyush-kurur I spoke to a team member and we found a workaround :)

Please re-open this bug and remove the wontfix tag. I am going to upload your git HEAD to experimental just as soon as some downtime in our infrastructure is resolved.

piyush-kurur commented 7 years ago

@spwhitton you (and the other team member) are awesome. But please feel free to drop it if you are unnecessarily burdened.

spwhitton commented 7 years ago

I started preparing the upload but I found that your repo now contains many PDFs. I have to confirm that these are compatible with the Debian Free Software Guidelines before I can upload. This will take some time (unless you want to remove the PDFs from the repo).

piyush-kurur commented 7 years ago

These are pdfs corresponding to various documentations of crypto primitives. I think it has the FIPS standard. Since you have been using the cabal tarball sofar you did not have this problem. I will try to replace it with plain text documentation of if there are no replacements I will move it to a separate repository. Coming to think of this, I am not even sure of the licensing of these pdf although are available for free on the internet. So yes these needs to go. I will open an issue particularly for this.

This brings back to another possible issue. There is a lot of stale code on github that are no more relevant but is there so that I can port some of these. It might be a good idea to get these things out.

piyush-kurur commented 7 years ago

@spwhitton I think I have got rid of all pdf files (and much more). So you can try uploading the head now. The latest release candidate is on the release-0.2.0 branch. Do you want be to merge it into master ? Meanwhile there is a build failure on travis that is not so serious (stack nightly) and windows (that probably needs to be worked on).

spwhitton commented 7 years ago

On Mon, Aug 14 2017, Piyush P Kurur wrote:

@spwhitton I think I have got rid of all pdf files (and much more). So you can try uploading the head now. The latest release candidate is on the release-0.2.0 branch. Do you want be to merge it into master ?

Thanks. For this test upload, it doesn't matter.

-- Sean Whitton

piyush-kurur commented 7 years ago

Okey I think most of the problems then are fixed on this end. I will leave this ticket open and just before the release of 0.2.0 we can review it.

Would you like to maintain some scripts that will help you with making the package easily on debian as part of the repo? I mean mostly things that are specific to raaz as opposed to generic haskell packages.

spwhitton commented 7 years ago

On Tue, Aug 15 2017, Piyush P. Kurur wrote:

Okey I think most of the problems then are fixed on this end. I will leave this ticket open and just before the release of 0.2.0 we can review it.

Thank you for caring about this!

I took a look at your HEAD. It still contains the HMAC RFC, which is not DFSG-free (see https://wiki.debian.org/NonFreeIETFDocuments). And raaz-cipher contains some PDFs and some ecryptTestData. But as you say, this doesn't matter for Debian since we can work from tarballs from Hackage.

Using the workaround I found, I've uploaded the original tarball from hackage to experimental: https://buildd.debian.org/status/package.php?p=haskell-raaz&suite=experimental

Would you like to maintain some scripts that will help you with making the package easily on debian as part of the repo? I mean mostly things that are specific to raaz as opposed to generic haskell packages.

What were you thinking such scripts would do?

-- Sean Whitton

piyush-kurur commented 7 years ago

On Tue, Aug 15, 2017 at 06:57:14PM +0000, Sean Whitton wrote: [snip]

I took a look at your HEAD. It still contains the HMAC RFC, which is not DFSG-free (see https://wiki.debian.org/NonFreeIETFDocuments).

I will get that that removed as well.

And raaz-cipher contains some PDFs and some ecryptTestData.

Are you looking at the branch release-0.2.0 or master ? I am collecting the release specific changes in the release-0.2.0 branch. In this branch the raaz-cipher directory does not exist. I will go ahead and merge this with master and please let me know how things go.

Using the workaround I found, I've uploaded the original tarball from hackage to experimental: https://buildd.debian.org/status/package.php?p=haskell-raaz&suite=experimental

Would you like to maintain some scripts that will help you with making the package easily on debian as part of the repo? I mean mostly things that are specific to raaz as opposed to generic haskell packages.

What were you thinking such scripts would do?

One specific example that I was thinking is that of the man page. There is a make file that builds the manpage from the markdown source using pandoc. So just uploading the tar ball might not work. You will need to do a make. To make this easier, you may want to add an appropriate target in the make file to copy this manpage to the right location.

Any such scripts/modification that you want todo which is raaz specific can go here. The idea is to make the debian packaging easier.

Regards,

ppk

piyush-kurur commented 7 years ago

I have pushed all the pending changes in the release-0.2.0 branch to master.

piyush-kurur commented 7 years ago

Also got rid of hmac rfc. Now the head should be okey for debian release.

piyush-kurur commented 7 years ago

There was a word size confusion in the blake2s implementation and as a result I guess it was failing on a lot of architectures. I have pushed what I believe is a fix.

spwhitton commented 7 years ago

On Wed, Aug 16 2017, Piyush P Kurur wrote:

And raaz-cipher contains some PDFs and some ecryptTestData.

Are you looking at the branch release-0.2.0 or master ? I am collecting the release specific changes in the release-0.2.0 branch. In this branch the raaz-cipher directory does not exist. I will go ahead and merge this with master and please let me know how things go.

Oh, sorry, I was looking at master.

One specific example that I was thinking is that of the man page. There is a make file that builds the manpage from the markdown source using pandoc. So just uploading the tar ball might not work. You will need to do a make. To make this easier, you may want to add an appropriate target in the make file to copy this manpage to the right location.

Any such scripts/modification that you want todo which is raaz specific can go here. The idea is to make the debian packaging easier.

Don't worry -- Debian already has tools to do this sort of thing. Once you release I'll investigate, but I think we'll be fine.

On Wed, Aug 16 2017, Piyush P Kurur wrote:

There was a word size confusion in the blake2s implementation and as a result I guess it was failing on a lot of architectures. I have pushed what I believe is a fix.

Would you like me to upload rc2?

-- Sean Whitton

piyush-kurur commented 7 years ago

On Wed, Aug 16, 2017 at 06:10:02PM +0000, Sean Whitton wrote:

On Wed, Aug 16 2017, Piyush P Kurur wrote:

There was a word size confusion in the blake2s implementation and as a result I guess it was failing on a lot of architectures. I have pushed what I believe is a fix.

Would you like me to upload rc2?

Yes that would be good. The Debian build is what indicated this problem. If this upload goes well, I think I will release.

Regards

ppk

spwhitton commented 7 years ago

Hello Piyush,

On Wed, Aug 16 2017, Piyush P Kurur wrote:

Yes that would be good. The Debian build is what indicated this problem. If this upload goes well, I think I will release.

Could you provide me with the tarball that doesn't include the extraneous files, please? I.e. like the previous RC.

While it is possible for me to work from your release-0.2.0 branch, that would require me to throughly check all new files for redistributibility. This seems like a waste of time if they will not be included in the final release of 0.2.0.

-- Sean Whitton

piyush-kurur commented 7 years ago

With the commit 04d3e0d00, I have removed all documentation. The master and release-0.2.0 are now in sync and I have uploaded the current head on hackage as a candidate release.

So as of now you could choose either of the following for rc2.

  1. The HEAD of the master branch (in case you have some automation there). You can get a tarball directly from github. I believe this should be available from https://github.com/raaz-crypto/raaz/archive/master.tar.gz

  2. The Candidate upload at https://hackage.haskell.org/package/raaz-0.2.0/candidate

I think the candidate package is better but if the debian system can manage direct upload from head that would also be good. That way you are saved of a lot of work.

spwhitton commented 7 years ago

Done. Build logs will be in the same place. Hope it's useful.

I normally base my Debian packaging on upstream git branches but there are two barriers to doing that for raaz:

  1. Your HEAD contains the source for several different Hackage packages.

  2. The Haskell team's tools are very tightly tied to Hackage and its tarballs.

-- Sean Whitton

piyush-kurur commented 7 years ago

On Fri, Aug 18, 2017 at 09:52:24PM +0000, Sean Whitton wrote:

Done. Build logs will be in the same place. Hope it's useful.

I normally base my Debian packaging on upstream git branches but there are two barriers to doing that for raaz:

  1. Your HEAD contains the source for several different Hackage packages.

They exist no more. I believe I removed all of it in commit c19e94227f8423433747fb97f4fe7a1eeb1ffeff

  1. The Haskell team's tools are very tightly tied to Hackage and its tarballs.

I can keep uploading the candidates. I realise that something is not working on kfreebsd and hurd variants which I will need to fix.

Regards,

ppk

piyush-kurur commented 7 years ago

@spwhitton I have uploaded one more candidate release which hopefully will fix the build failure on hurd and kfreebsd platforms. As I said in the previous reply, you can choose to upload from the candidate release tarball or can use the HEAD of the git repository if that is better automated (I believe that I have fixed the issues of multiple packages).

Thanks for all the hard work.

spwhitton commented 7 years ago

Done!

-- Sean Whitton

piyush-kurur commented 7 years ago

Okey now that all the builds succeed, I am ready for release. Howerver, there is a small issue. If you are using the hackage tar ball then it will not have the man page. This is because the tarball is built using either that cabal/stack sdist command. I can make this happen by including the man page (which is in markdown) together as extra-source but I am not sure if that is the best way to do this. Note that the tarball from git will not have this problem.

piyush-kurur commented 7 years ago

I will be adding the man page source in the cabal extra field. So I think this is resolved. Waiting for the final travis build before release.

piyush-kurur commented 7 years ago

Okey I have done the release. Please upload. Thanks for all the effort.

spwhitton commented 7 years ago

Debian unstable tracks Stackage, so it'll be a little while before raaz 0.2.0 makes it in. Congratulations on your release, anyway!