Bitcoin 0.10 has been released - what should we do?

[zathras-crypto]

Hi all,

I'd like to get a discussion going with as much participation as possible. The issue is simply that Bitcoin 0.10 has been released, and this is not compatible with OmniCore 0.0.9.1 (due to headers first).

We're gearing up to release the UI, however we need a strategy to handle the issue. We can't have our first major end user release being a failure because we haven't planned for this appropriately.

Thoughts please?

Thanks Z

[zathras-crypto]

Right right that makes much more sense, thanks for the clarification :)

[dexX7]

Fine, so what do you suggest to do now? :)

[zathras-crypto]

As next step, one of the owners, namely @CraigSellars or @achamely, would have to copy/clone/mirror the current mastercoin-MSC/mastercore to OmniLayer/omnicore

Adam, could you do the honours mate?

[zathras-crypto]

In case I wasn't clear guys this is ready to go - I've checked the mastercore_* files were unchanged and the subsequent commits were clean - it's literally just waiting on the repository move @CraigSellars @achamely :) Thanks Z

[zathras-crypto]

Just a little chase :) can we get this repo move actioned? (I'm working locally atm)

[dexX7]

Well yeah, would be great. :) Please publish/push nevertheless. :)

[zathras-crypto]

Heh, not so sure I should publish what I have right now lol it's a bit of a mess :P (I'm in the middle of redoing the UI for 0.0.10)... As a side note, how would I do that on github (when I'm ready to push)? Right now I'm working on a new branch I checked out dexx-0.0.10-ui directly under a clone of your bitcoin repo with the origin as origin/omnicore-0.0.9.2-no-history - I'm just using it as a base for the UI work but if I did want to push I'd need to I think add zathras-crypto as a remote and then push to that?

[dexX7]

Ah I see. :)

I usually start with a new branch, based on the current tip. Let's assume upstream points to the Omni Core repo, where omnicore-0.0.10 is the main branch, which you already cloned locally.

To switch to that branch:

git checkout omnicore-0.0.10

Refresh your local copy and pull updates, if there are any:

git fetch upstream omnicore-0.0.10 << might not be necessary
git pull upstream omnicore-0.0.10

You probably already have done something similar, but with my branch. From here:

Add your remote:

git add remote public https://github.com/zathras-crypto/mastercore.git

Create a new branch:

git checkout -b ui-feature-branch

Do some work, add some commits.

And finally push:

git push public ui-feature-branch

To remove a branch locally:

git branch -D temp-branch-to-delete

public and ui-feature-branch can of course be replaced by something else.

Hope this helps. :)

[achamely]

Just catching up on this thread now. We have to options, I can either Move the existing repository from mastercoin-msc to omnilayer and rename it to omnicore or i can clone a new instance of mastercoin-msc and rename it to omnicore. Suggestions/thoughts @zathras-crypto @CraigSellars (Info about moving a repo and impact is available here https://help.github.com/articles/transferring-a-repository/)

[zathras-crypto]

My preference would be to clone rather than move :)

[zathras-crypto]

@dexX7 just a quick update mate - @achamely & I spoke about this on Skype today - the decision is essentially one of move or clone - will be discussed at the meeting in the morning and then can get taken care of :)

Thanks Z

[dexX7]

Great to hear.

[zathras-crypto]

Sean mentioned in the meeting today that it might be better to not fork from Bitcoin and just do a fresh push to the new repo from a local clone, as this helps with certain stats? Any feedback on that @dexX7?

Otherwise it'll be a new fork of Bitcoin, not a move of the old repo.

Thanks :) Z

[achamely]

After discussing in the meeting we are going with a fresh fork of bitcoin to preserve upstream visibility and allow changes to be pulled/pushed? in future. its up on https://github.com/OmniLayer/omnicore now.

[dexX7]

After discussing in the meeting we are going with a fresh fork of bitcoin to preserve upstream visibility and allow changes to be pulled/pushed?

Thanks. This would have been the point I'd raise as well.

[dexX7]

@achamely: given that Bitcoin Core 0.10 works very well with Travis and the base file already exists - even though it fails with Omni Core due to the wallet and txindex requirement at the very moment - I suggest to enable Travis for OmniLayer/omnicore. PR/build notifications could be turned off at first.

[dexX7]

Ah, and please enable "issues". :)

[achamely]

Issues enabled (as well as wikis)

[achamely]

@dexx i know @msgilligan has already got a jenkins ci environment setup. How is travis different/better?

[dexX7]

@achamely: It's not better, but what I really miss are automated tests of pull requests.

Doing consensus tests and alike should still be done via the CI server - it's more about using what is already given, and leveraging the test infrastructure that comes with Bitcoin Core. In particular this includes testing of ARM, Linux, Windows and OS X builds with different configurations (example).

Depending on whether this is an option, I'd take a look at how the "txindex" and "with wallet only" build requirements could be addressed (during testing), which would currently result in test failures.

[msgilligan]

Jenkins supports automated tests of pull requests, but there are security and configuration issues in setting it up and I haven't had time to resolve them.

[dexX7]

@msgilligan: sorry to be unclear, this is acutally the point: it is costly, in terms of setup time, risk exposure and additional load on the server, which is better be used for the core operation - deep testing of Omni Core.

It should probably be aimed for to provide a similar set of features and tests via the CI environment, but until then I actually see no reason not to use Travis as complementary.

https://github.com/OmniLayer/omnicore/pull/4 tackles the issues related to txindex and wallet requirements, which I mentioned earlier.

[msgilligan]

I think we're in agreement. Absolutely use Travis as a complementary solution. However at some point if I find an easy way to combine Docker and Jenkins -- or need to do it for another reason -- then we can revisit this.

[dexX7]

Ah thanks @msgilligan. Ideally we would run both test suits, to ensure all builds are fine and pass the Omni Core related tests. I'm running Travis on my bitcoin/omni repo for a while and yesterday noticed it's already pretty close to the maximum build time of 50 minutes, so I guess there is no way to squeeze it in as well. If it turns out those 50 minutes are not sufficient, I even guess the number of tests would have to be reduced. Unfortunally Travis doesn't allow caching for "non container builds", but using "container builds" is currently not possible, because sudo (and thus apt-get install) is not allowed there.

@zathras-crypto can you please take a look at https://github.com/OmniLayer/omnicore/pull/4? (edit: merged) @achamely: once that PR is in, these are the steps to take: https://github.com/mastercoin-MSC/mastercore/pull/269#issuecomment-68613936

[dexX7]

@msgilligan: Soo.. I reached out to the Travis support and asked, if it might be possible to whitelist certain repos for caching (as appearingly done for bitcoin/bitcoin), but this was basically declined. The speedup would roughly be 5-6x, and I guess it would be possible to manually cache some directories, but this is, at least for me, be not trivial to realize.

But I also played around a little with cross compiling on my Ubuntu VM, and this part was actually easier than anticipated.

In another fresh VM I then setup Jenkins and created a job with multi-configuration build matrix. Turned out this works different than expected, but I think I got it now. The key was to have only one build matrix parameter, named compiler (or whatever), with the values:

"ARM" "bitcoind" "64-bit_qt" "32-bit_dash" "Cross-Mac" "Win64" "Win32"

As far as I can see it's not very different to standard jobs, except the build matrix:

For the actual build step, I converted the Travis configuration into a shell script:

• https://gist.github.com/dexX7/b0ef8a1c515c73103839

There were a few issues so far:

• [x] not surprising, it was required to install git and the dependencies listed in doc/build-unix.md
• [x] further, the user jenkins requires sudo to fetch and install apt packages
• [x] building must be done sequentially, because otherwise there seem to be file locking issues
• [x] running the deamon of the Windows builds was sort of unsuccessful, so testing those is disabled for now
• [ ] and last but not least, for OS X building some SDK is required (I guess that's the same one I was emailing you about earlier), which is hosted on bitcoincore.org, and I never changed it, because my Travis builds were just fine, but it looks like certain access is whitelisted and downloading the SDK from another source fails

I'm currently running this configuration and not all builds with spock tests finished, but the Windows builds (with Bitcoin Core tests) appear to work well, and the spock tests I saw so far, too. :)

What do you think about it?

Re: pull requests and status updated, which I posted earlier, but just in case:

• https://github.com/msgilligan/bitcoin-spock/issues/33#issuecomment-74153263

Edit: even though I'm testing building via a build matrix, I assume it's more convenient in terms of test output handling, when using more than one job.

---

@achamely: I suggest to enable Travis nevertheless, even though this can only be used for building and Bitcoin Core tests, it's an improvement, basically without any effort. :)

[msgilligan]

Hi @dexX7 I'm 100% focused on OmniJ for the next couple of days. I'd love to see cross-compiling on Jenkins, though.

[dexX7]

@achamely @CraigSellars (as organization admins):

Please enable Travis CI for automated testing, including pull requests for OmniLayer/omnicore. While this doesn't cover the RPC tests via bitcoin-spock, it does test six different build plattforms, a huge number of Bitcoin Core tests, and of course the unit tests.

Given the increased number of unit tests currently added, this becomes more and more relevant.

It takes 5 minutes and simply consists of a GitHub authentification. See: https://github.com/mastercoin-MSC/mastercore/pull/269#issuecomment-68613936

[dexX7]

We have now a Bitcoin Core 0.10 based branch over here: https://github.com/OmniLayer/omnicore/tree/omnicore-0.0.10

This can be closed in my opinion.