Bugfix/packaging not including some jars

rodvar commented 2 weeks ago

Purpose

Users report cli and daemon binaries are not included anymore in bisq release
Fixes #7235

Approach

Release scripts seem a bit abandoned, it was not even compiling. Fix that and also the dirs relativity as well as docs to be able for any release manager to execute scripts on terminal and get the release files.

Dev Logs

 - fix compilation issue on package.gradle to be able to generate
   binaries (packageInstallers task)
 - fix directory management in finalize.sh release script
 - updated docs with important notes for release managers (**finalise.sh should be run always**)
 - fix docs to note that the finalize script needs to be run for every
   platform
 - ensure cli and daemon and built -> cli and daemon binaries get generated and included in the release now

Screenshot from 2024-09-17 16-54-15

boring-cyborg[bot] commented 2 weeks ago

Thanks for opening this pull request!

Please check out our contributor checklist and check if Travis or Codacy found any issues with your PR. Also make sure your commits are signed, and that you applied Bisq's code style and formatting.

A maintainer will add an is:priority label to your PR if it is up for compensation. Please see our Bisq Q1 2020 Update post for more details.

HenrikJannsen commented 2 weeks ago

@alvasw Could you review that as well?

orangefren commented 2 weeks ago

Thank you for taking the time to work on this issue <3

Release scripts seem a bit abandoned, it was not even compiling

I'm a little confused by this. If the release scripts were broken, yet releases were being released.....then HOW were they being made? Wouldn't this indicate whoever produces the binaries for release does so in their own, undocumented, unreproducable, unauditable way?

rodvar commented 2 weeks ago

Thank you for taking the time to work on this issue <3

Release scripts seem a bit abandoned, it was not even compiling

I'm a little confused by this. If the release scripts were broken, yet releases were being released.....then HOW were they being made? Wouldn't this indicate whoever produces the binaries for release does so in their own, undocumented, unreproducable, unauditable way?

Thank you @orangefren , and thanks also for your interesting comment.

To be fair, the only non compiling script was package.gradle, and whilst I was working on this it got removed (I know cause I have to rebase my branch to catch the changes and fix conflicts last minute). Then the bash script finalize.sh has some logic issues that caused the problem related to daemon and cli jars which originated this PR as per users request.

That said, the docs were a bit confusing and even if the release managers have their own perfected local script based on the ones in the codebase, they could easily miss some stuff.

IMHO every contributor should be able to generate a release, but the release that actually gets published needs to have some limitations on who can publish it and that's handled with the concept of maintainers (people who started bisq originally and have a lot of stake in the game)

Probably @HenrikJannsen or @alejandrogarcia83 or @alvasw can shed some light on this..?

HenrikJannsen commented 2 weeks ago

Thank you for taking the time to work on this issue <3

Release scripts seem a bit abandoned, it was not even compiling

I'm a little confused by this. If the release scripts were broken, yet releases were being released.....then HOW were they being made? Wouldn't this indicate whoever produces the binaries for release does so in their own, undocumented, unreproducable, unauditable way?

I am not very familiar with current Bisq 1 changes related to release process. But as there have been larger changes with a BitcoinJ bug [1] and the way how Tor is integrated [2] I assume some things have not got updated in the docs and some pending changes have not been made it to a PR for the build system yet.

Bisq has no deterministic build, only the jar file is deterministic. To make the binaries themself deterministic is an old but unresolved issue (and not trivial).

[1] BitcoinJ had a bug with a field holding the accumulated proof of work which was designed too small. They made a quick fix to change the field from signed to unsigned to win a bit of time. This is already in Bisq as well. The more solid change was made in upstream BitcoinJ and @alvasw has worked to rebase our branch on that, which was a big effort as our branch was quite old. As far I am aware he is now in the testing phase and is re-using Bisq2 test infrastructure with regtest bitcoind and those changes had impact on gradle tasks the build system I assume (not followed closely).

[2] @alvasw also has worked to use the more solid Bisq2 infrastructure for downloading, verifying and zipping the Tor binaries for Bisq 1. As above I assume those changes might have caused issues with the build process for binaries. It is very time consuming to test the binary build as you need to do that on all the supported OS. As it is usually only the release manager who use the binary build path, thus bugs in that area are often detected late and docs are not that well maintained.

orangefren commented 2 weeks ago

In light of these, and other, issues we've decided to launch an initiative with an initial funding of 5 BTC to make a better Bisq 1 client

The initiative is posted here: https://bitcointalk.org/index.php?topic=5509573.new#new

HenrikJannsen commented 1 week ago

In light of these, and other, issues we've decided to launch an initiative with an initial funding of 5 BTC to make a better Bisq 1 client

The initiative is posted here: https://bitcointalk.org/index.php?topic=5509573.new#new

I read your initiative at bitcointalk and welcome your motivation but would like to add some information to avoid misconceptions and misinterpretations.

First ot all, many of those issues you posted are well know to the Bisq contributors and the result of that was the efforts on Bisq 2 [1]. Bisq 2 is a complete new code base with lots of improvements and most importantly with the concept of multiple trade protocols. It has currently only 1 trade protocol, Bisq Easy, which is primarily designed for no-coiners and newbies and has different trade-offs compared to the Bisq 1 Multisig protocol (though it turned out that due its "easyness"and flexibility it attracts not only newbies). It has to be seen as complementary to Bisq 1 and not as competitor.

Work on the MuSig protocol (will become the replacement for Bisq 1 MultiSig) will start soon and there is a solid concept [2] in place which comes with lots of improvements compared to Bisq 1, most notable the reduction of 4 transactions involved to one, next to privacy and security improvements.

Other trade protocols like Lightning-Fiat or Monero swaps are on the roadmap and described in the app's protocol overview.

I think a big misconception in your criticism is the role of Java. The issues in Bisq 1 have little (to nothing) to do with the choice of language/runtime but with lack of dev resources and some conceptual scaling issues as well as the fact the Bisq 1 is about 10 years old code base with related wear from that.

Your idea to build a Bisq alternative based on Electrum is not reflecting the core challenges when building a DEX. The wallet is the easiest part. The 2 core elements which enables Bisq's decentralized model are the P2P network and the DAO.

The importance of the P2P network should be obvious, and to build such is not a trivial task and Electrum has nothing to offer in that regards.

The less obvious part might be the DAO. Many do not appreciate and understand the importance of the DAO for the trade protocol. Without the DAO the Bisq MS protocol would fall back to the model Bisq used initially, to have a 3rd party as part of a 2of3 Multisig which comes with security and regulatory risks. See Haveno as example how they suffer from not finding candidates who are trusted and have built up a solid reputation so users have confidence that the arbitrator does not scam them and who are willing to take that considerable risk from legal uncertainties as this entity is part of the trade to some extent and depending on legal interpretation could be held liable or at least subject to regulations.

Beside that, the DAO is the funding model and without it it would be even harder to find competent developers and other contributors.

Your 5 BTC bounty is very generous, but does not reflect the reality of professional software development costs. A highly skilled and experienced developer in the West costs about 8-15k USD/month. An optimal team size would be at least 3 developers. So 5 BTC would be spent in less than a year. See Haveno how long it took them to just make a clone of Bisq by forking it (not building it new from scratch). Bisq2 was also a multi year effort.

We are aware that we lack on communications in many of those areas as well as on devs to get all those things done we would like to. So if you want to improve Bisq 1 or help to make Bisq 2 MuSig protocol a reality faster, send us competent Java devs. Money is not the problem, the shortage of skilled and experienced devs is.

I just saw that line in your post: "This project is not related to Bisq 2 thus any mentions of it will be considered offtopic" This makes me a bit sick to be honest. If there is no openness and willingness to try to understand why those who are most experienced with Bisq decided to build a new version to fix the problems of Bisq 1, then any further discussion is just waste of time. Let me know if you stick to that statement, as in that case I have better investment of my time.

[1] https://bisq.network/blog/bisq-2-now-in-beta/ [2] https://github.com/bisq-network/proposals/issues/456

orangefren commented 1 week ago

Thank you taking the time to respond so thoroughly.

First ot all, many of those issues you posted are well know to the Bisq contributors and the result of that was the efforts on Bisq 2 [1]. Bisq 2 is a complete new code base with lots of improvements and most importantly with the concept of multiple trade protocols. It has currently only 1 trade protocol, Bisq Easy, which is primarily designed for no-coiners and newbies and has different trade-offs compared to the Bisq 1 Multisig protocol (though it turned out that due its "easyness"and flexibility it attracts not only newbies). It has to be seen as complementary to Bisq 1 and not as competitor.

Work on the MuSig protocol (will become the replacement for Bisq 1 MultiSig) will start soon and there is a solid concept [2] in place which comes with lots of improvements compared to Bisq 1, most notable the reduction of 4 transactions involved to one, next to privacy and security improvements.

Other trade protocols like Lightning-Fiat or Monero swaps are on the roadmap and described in the app's protocol overview.

I just saw that line in your post: "This project is not related to Bisq 2 thus any mentions of it will be considered offtopic" This makes me a bit sick to be honest. If there is no openness and willingness to try to understand why those who are most experienced with Bisq decided to build a new version to fix the problems of Bisq 1, then any further discussion is just waste of time. Let me know if you stick to that statement, as in that case I have better investment of my time.

I'm not opposed to the development of Bisq 2. Our initiative is to build something that's as simple and lightweight as possible. As for the "Bisq 2 mentions will be considered offtopic" - we want to make it clear what we're building is not Bisq 2 and simply want to avoid responses to the tune of "Bisq 2 will make your work obsolete" (which it won't).

Of course discussing features from the Bisq 2 roadmap, such as MuSig (I greatly appreciate the research that went into it) and looking for ways to implement them in our light client is perfectly fine and encouraged, especially that it's a clear upgrade to the original trading protocol.

Your 5 BTC bounty is very generous, but does not reflect the reality of professional software development costs. A highly skilled and experienced developer in the West costs about 8-15k USD/month. An optimal team size would be at least 3 developers. So 5 BTC would be spent in less than a year. See Haveno how long it took them to just make a clone of Bisq by forking it (not building it new from scratch). Bisq2 was also a multi year effort.

The bounty is just to kickstart development :)

I think a big misconception in your criticism is the role of Java. The issues in Bisq 1 have little (to nothing) to do with the choice of language/runtime but with lack of dev resources and some conceptual scaling issues as well as the fact the Bisq 1 is about 10 years old code base with related wear from that.

We are aware that we lack on communications in many of those areas as well as on devs to get all those things done we would like to. So if you want to improve Bisq 1 or help to make Bisq 2 MuSig protocol a reality faster, send us competent Java devs. Money is not the problem, the shortage of skilled and experienced devs is.

I believe your choice of Java is a key reason why you struggle to find competent, skilled and experienced devs.

bisq-network / bisq