search

pact-foundation / pact-ruby-standalone

A standalone pact command line executable using the ruby pact implementation and Travelling Ruby
https://pact.io
MIT License
39 stars 32 forks source link

chore: upgrade to travelling ruby 2.4.10 #75

Closed trammel closed 2 years ago

trammel commented 2 years ago

Travelling ruby has deprecated support for linux-x86 (x86_64 is still supported), so that's been removed from the created packages.

The naming convention for the Windows client has also changed, and that's fixed.

Tests are passing. This addresses https://github.com/pact-foundation/pact-ruby-standalone/issues/63

Lewiscowles1986 commented 2 years ago

I was looking into this as a possible root cause of some issues. While upgrading is always better than leaving as-is. Does anyone know if it might be easier to just release as a standard ruby gem, and install ruby native to each docker container (so that further upgrades, such as 2.7, and even 3.x will be possible)?

trammel commented 2 years ago

I think the point of this project was to release a single package that could be natively executed on different platforms.

I believe the maintainers want to continue releasing standalone binaries, but using Rust in the future https://github.com/pact-foundation/pact-ruby-standalone/issues/70#issuecomment-967725063

However with the author of Traveling Ruby explaining that there won't be any future releases, https://www.joyfulbikeshedding.com/blog/2021-01-06-the-future-of-traveling-ruby.html I think this is the last version of Ruby that can be supported using this method.

bethesque commented 2 years ago

This PR is the best thing since sliced bread! (apart from having some completely new, better and properly supported way of doing this 😆)

The only problem I can see is that the name change to the windows package is going to break the downstream libraries that are currently expecting to unzip a package with the old name. Before we merge this and set the Rube Goldberge release machine in process, we need to work out how we'll handle that.

bethesque commented 2 years ago

Given we don't have to disambiguate between two different windows packages, I think the easiest approach would be to keep the windows package name the same by passing in an extra parameter to the create_package method that is used in preference to the target in the package name when it is set.

YOU54F commented 2 years ago

Epic @trammel, I was going to say before I @bethesque had commented, that this would make her very happy. It's the best thing since sliced bread. 😂

The last time she said that I think was when she first met Pact

Soon after, Beth decided that Pact idea was the best thing since sliced bread, and she hasn't stopped yacking on about it since.

Also very cool to see, looking at your profile that you are from Rea (the birthplace of Pact), and a look to be a seasoned ruby pro.

Pact was originally written by a development team (including Ron Holshausen from DiUS) at realestate.com.au in 2013. They were trying to solve the problem of how to do integration testing for their new Ruby microservices architecture, and they threw together a codebase that was to become the first Pact implementation.

Taken from the page, where both those quotes from.

https://docs.pact.io/history

I may be showing you stuff you already know, but for me its total genesis-ception 🤯 😂

YOU54F commented 2 years ago

Also whilst we are looking at travelling ruby, i was checking out some active forks

https://gist.github.com/YOU54F/b250c64ff51d60356cd8aa845602f337

this person has just forked it and has a 2.7 and 3.1 branch 😌 , not sure if they work but cool to see a bit of other people interested in getting it working.

https://github.com/cloudaware/traveling-ruby/tree/2.7.6

YOU54F commented 2 years ago

Linking this issue as it seems relevant to the conversation, should we add in tests for 2.4 or would they be redundant as they would be tested via this repo where 2.4 is used.

https://github.com/pact-foundation/pact_broker-client/issues/111

what if someone wanted to run the gem directly on ruby 2.4, should be advise that they should upgrade to 2.7 as versions below are EOL

Thanks for all this, we really appreciate it as I am sure the community will!

bethesque commented 2 years ago

Each of the gems included should be running tests on 2.2 (which is now 2.4) to make sure they pass before publishing the new gem. The pact broker client 2.2 tests have been failing for a while because the environment can no longer be set up properly in the build because 2.2 is so old, and I hadn't had time to fix them up. 2.4 should be fine though now.

It an ideal world, we could add some smoke tests for the package, but I think there are probably other things higher on the priority list for now.

People who want to run the gems with ruby will install the specific ruby gem directly. The amalgamated standalone is only necessary because we're repackaging it.

trammel commented 2 years ago

Given we don't have to disambiguate between two different windows packages, I think the easiest approach would be to keep the windows package name the same by passing in an extra parameter to the create_package method that is used in preference to the target in the package name when it is set.

jonpad@jonpad-Laptop:~/git/pact-ruby-standalone (chore/upgrade-to-ruby-2-4-10)$ ls -la pkg/
total 37192
drwxr-xr-x  2 jonpad jonpad     4096 May 24 19:42 .
drwxr-xr-x 10 jonpad jonpad     4096 May 24 19:42 ..
-rw-r--r--  1 jonpad jonpad  7960869 May 24 19:42 pact-1.88.88-linux-x86_64.tar.gz
-rw-r--r--  1 jonpad jonpad  8296395 May 24 19:42 pact-1.88.88-osx.tar.gz
-rw-r--r--  1 jonpad jonpad 10905078 May 24 19:42 pact-1.88.88-win32.zip
-rw-r--r--  1 jonpad jonpad 10905078 May 23 14:00 pact-1.88.88-x86_64-win32.zip

I've updated the PR, and the newer generated packages is now just suffixed 'win32.zip'. Does that help?

trammel commented 2 years ago

Also whilst we are looking at travelling ruby, i was checking out some active forks

https://gist.github.com/YOU54F/b250c64ff51d60356cd8aa845602f337

this person has just forked it and has a 2.7 and 3.1 branch relieved , not sure if they work but cool to see a bit of other people interested in getting it working.

https://github.com/cloudaware/traveling-ruby/tree/2.7.6

Possibly, but in all honesty, I'm only submitting this PR so that pact-ruby-standalone moves on to ruby 2.4. Which would allow pack_broker-client to move on to at least ruby 2.4. Which would allow pact_broker-client's gemspec to upgrade to a more recent version of httparty. Which would allow upgrades to the version of pact_broker-client in company gems. Which would allow renovatebot to upgrade some codebases at REA. Which would mean next quarter's system health check, those codebases won't turn "red". Which would mean my annual bonus is less likely to be affected.

trammel commented 2 years ago

Also very cool to see, looking at your profile that you are from Rea (the birthplace of Pact), and a look to be a seasoned ruby pro.

I've worked with Ron (a true legend) way back, but I was out of country when Beth started Pact while at REA. I did attend a couple of her talks and training sessions though :smile: It's still used in quite a few places, and still marked as "Adopt" on REA's Tech Radar.

bethesque commented 2 years ago

Man, that's some yak shaving required! I would really like to pull out HTTParty and replace it with Faraday (most of the calls use Faraday, but the original ones are still on HTTParty), but it's never been the top of the priority list compared to all the other things that need doing.

bethesque commented 2 years ago

When I ran bundle exec rake package locally, and then extracted the mac package and tried to run the pact command, I got the following error. It may be because I've just set up my new M1 mac and there might be issues, but can you try it on your machine and see what happens Jonathon?

Screen Shot 2022-05-25 at 11 11 11 am
trammel commented 2 years ago

When I ran bundle exec rake package locally, and then extracted the mac package and tried to run the pact command, I got the following error. It may be because I've just set up my new M1 mac and there might be issues, but can you try it on your machine and see what happens Jonathon?

I get exactly the same problem. I'll dig a little. I have vague memories of the JSON gem being extracted from stdlib and being required in Gemfiles.

trammel commented 2 years ago

Okay. I don't know why bundler isn't finding 'json'. It's a bit difficult because the 'json' gem uses native extensions, which really goes against the whole idea of cross-platform binaries.

So there's two branches to pick from

1) Package the 'json' gem with native extensions provided by Traveling Ruby.

In both cases bin/pact runs on Linux and displays a help message. I don't know how well either works on Mac (in both options) or Windows (in the second option).

bethesque commented 2 years ago

A much as I'd love to drop support for Windows, I think our many Windows users might have other thoughts on that issue 😆

If you put option 2 into the PR, I can try building it on my machine and see how it goes.

trammel commented 2 years ago

A much as I'd love to drop support for Windows, I think our many Windows users might have other thoughts on that issue laughing

The sooner you can find a solution for Traveling Ruby, the better.

I've cherry-picked the commit across, you should be good to test it.

bethesque commented 2 years ago

The ruby standalone is being replaced by a Rust shared core. Most of the main languages are in the process of converting from the ruby to the rust.

bethesque commented 2 years ago

Ok, this builds and runs for me now.

Screen Shot 2022-05-26 at 10 59 25 am

pact-1.88.89-win32.zip

I'll see if I can get someone on a Windows machine to verify the win32 package.

bethesque commented 2 years ago

So, Ron had this issue when running the new package, but given that someone has raised the same issue with the existing package, I think it's unrelated. I want to merge this, but I just need to get a comms plan together to make sure that each of the "wrapper library maintainers" can do some manual testing before we run the entire Rube Goldberg release machine.

bethesque commented 2 years ago

Note to self: this needs to go out before pact-support because a PR recently introduced a gem that can't run on 2.2

YOU54F commented 2 years ago

If the dep graph of doom is up to date, this is a good place to start for a comms plan for the maintainers.

https://github.com/pact-foundation/devrel/blob/master/dependency_graph.md

We have a matrix setup, to begin to test out the standalone, when it is released again the example repos. and I would like to build something out like that to test the example repos that use Pact implementations.

we can get Ron to check his version of windows, and I can check if that issue if affecting enterprise/pro editions or it just affected that user.

we could get a message like this, in each of the repos, pointing back to a central tracking issue in the devrel repo and I can aid people with a roll-out plan.

Screenshot 2022-05-31 at 02 10 10

As part of this, we can get a solid plan for testing which means as we upgrade things in the future, the Rube Goldberg machine becomes a safer beast. (well it's not unsafe, but reduce the burden on needing to manually verify things, just push a button, make a cuppa and come back and see what breaks)

I love the idea of being able to do canary builds that push out across and test on something like an insiders track, which is marked as unstable until everything passes

YOU54F commented 2 years ago

So you can trigger from here and enter the value of the standalone you want to test

https://github.com/pactflow/example-bi-directional-consumer-cypress/actions/workflows/test_pact_cli_tools_cross_os.yml

RC1 looks to have passed everything, RC2/3 look like corrupt packages, RC4 failing on windows

I quickly tested inbetween meetings, on macos (worked) and windows with RC4 and got the same behaviour as exhibited in the above CI runs

D:/a/example-bi-directional-consumer-cypress/example-bi-directional-consumer-cypress/pact/lib/app/pact-mock-service.rb:1:in `require': cannot load such file -- pact/mock_service/cli (LoadError)
    from D:/a/example-bi-directional-consumer-cypress/example-bi-directional-consumer-cypress/pact/lib/app/pact-mock-service.rb:1:in `<main>'
mingw32-make: *** [Makefile:176: install-pact-ruby-standalone] Error 1

using a mingw32 (git bash) shell

just checked in a cmd shell and got the same 😢

bethesque commented 2 years ago

Ok, I worked out what needed to be fixed for Windows. The PLATFORMS section of the Gemfile.lock needed to have x64-mingw32 added to it.

YOU54F commented 2 years ago

Winner winner chicken dinner - all working nicely

Cross os test run https://github.com/pactflow/example-bi-directional-consumer-cypress/actions/runs/2427684436

windows specific run using 1.88.91-RC5

https://github.com/pactflow/example-bi-directional-consumer-cypress/runs/6707355793?check_suite_focus=true

great work @trammel & @bethesque

trammel commented 2 years ago

Thanks to Beth and the team. It must have been a lot of work testing downstream systems and the varying affected different architectures.

bethesque commented 2 years ago

Yousaf has recently set up a bunch of builds to test All The Combinations luckily. Great work!

It took me all morning stepping though every line of bundler in the working and non working packages, using puts to debug, to work out what the problem was on Windows however! I'm so unfamiliar with dev on Windows. My fingers just don't know what to do because all my shortcuts don't work!

Many thanks for making this happen Jonathon. There's an unrelated build failure in pact js at the moment (which is what I assume you're using), but I hope we can get it out asap.