Missing open3 gem

[mattlindsey]

Adding a pdf document gives me the error below, which I resolved by adding gem "open3" to Gemfile. Ruby 3.1.4. Maybe adding require "open3" would also fix it.

jobs    | 2024-09-18T21:28:30.248Z pid=59558 tid=ohq WARN: NameError: uninitialized constant Parsers::Pdf::Open3
jobs    |
jobs    |       txt, stderr, status = Open3.capture3(cmd, stdin_data: @document.contents, binmode: true)

[nickthecook]

Cool, is it a PDF you can share?

I've not seen that error; assuming it has something to do with the file itself...

[mattlindsey]

I tried a few different pdfs and versions of ruby 3 and this PR fixed it #82, although I suppose it's possible it's unique to me.

Also, 35 of the tests are failing for me. Here are the first few:

(base) ➜  archyve git:(require-open3) ops test
Running `bundle exec rspec --tag "~skip"` in environment 'dev'...
fatal: No names found, cannot describe anything.
DEPRECATION WARNING: `Rails.application.secrets` is deprecated in favor of `Rails.application.credentials` and will be removed in Rails 7.2. (called from <top (required)> at /Users/mattlindsey/github/archyve/config/environment.rb:5)
Run options: exclude {:skip=>true}

Randomized with seed 55317
2024-09-18 19:19:47 INFO Selenium [:logger_info] Details on how to use and modify Selenium logger:                  |  ETA: 00:00:17
  https://selenium.dev/documentation/webdriver/troubleshooting/logging

2024-09-18 19:19:47 WARN Selenium [:selenium_manager] The chromedriver version (121.0.6167.85) detected in PATH at /usr/local/bin/chromedriver might not be compatible with the detected chrome version (128.0.6613.138); currently, chromedriver 128.0.6613.137 is recommended for chrome 128.*, so it is advised to delete the driver in PATH and retry

  1) models /v1/models returns 200
     Failure/Error: expect(call.code).to eq(200)

     NoMethodError:
       undefined method `code' for 401:Integer

             expect(call.code).to eq(200)
                        ^^^^^
     # ./spec/e2e/api/models_spec.rb:6:in `block (3 levels) in <main>'

  2) models /v1/models contains items
     Failure/Error: expect(call.parsed_body["models"]).not_to be_empty

     NoMethodError:
       undefined method `parsed_body' for 401:Integer

             expect(call.parsed_body["models"]).not_to be_empty
                        ^^^^^^^^^^^^
     # ./spec/e2e/api/models_spec.rb:10:in `block (3 levels) in <main>'

  3) models /v1/models returns a list of models
     Failure/Error: expect(call).to match_response_schema("models")

     NoMethodError:
       undefined method `match_response_schema' for #<RSpec::ExampleGroups::Models::V1Models "returns a list of models" (./spec/e2e/api/models_spec.rb:13)>
       Did you mean?  match_selector
     # ./spec/e2e/api/models_spec.rb:14:in `block (3 levels) in <main>'

[nickthecook]

Neat, I just added the end-to-end specs but initially didn't exclude them from a normal run of the unit tests. I've fixed that now, so ops test shouldn't invoke the e2e specs, and the errors you posted above should be fixed.

The e2e specs are a step towards automating testing but I still haven't solved the data setup, so I'm not sure they'll pass for everyone right now. However, I'm curious what ops e2e does on your machine!

[mattlindsey]

Cool. ops test is good now. ops e2e gives error below. Something with 'ollama'? ollama is running on my machine.

(base) ➜  archyve git:(main) ops e2e
Running `TEST_E2E=true bundle exec rspec --tag "~skip" spec/e2e` in environment 'dev'...
Run options: exclude {:skip=>true}

Randomized with seed 47028

  1) opp/api/tags returns models
     Failure/Error: response = HTTParty.get("#{endpoint}#{path}", headers:)

     Errno::ECONNREFUSED:
       Failed to open TCP connection to localhost:11337 (Connection refused - connect(2) for "localhost" port 11337)
     # ./vendor/bundle/ruby/3.1.0/gems/net-http-0.4.1/lib/net/http.rb:1603:in `initialize'
     # ./vendor/bundle/ruby/3.1.0/gems/net-http-0.4.1/lib/net/http.rb:1603:in `open'

[nickthecook]

Good to hear.

11337 is what Archyve's Ollama Proxy Port feature listens. It should be started by a line in Procfile.dev, but it sounds like it's not running. That should break more than one test, though - looks like there are 9 OPP tests:

$ ops e2e --tag opp
Running `TEST_E2E=true bundle exec rspec --tag "~skip" spec/e2e --tag opp` in environment 'dev'...
Run options:
  include {:opp=>true}
  exclude {:skip=>true}

Randomized with seed 10459
 9/9 |========================== 100 ===========================>| Time: 00:00:36

Finished in 36.89 seconds (files took 0.44331 seconds to load)
9 examples, 0 failures

Randomized with seed 10459

[nickthecook]

That should break more than one test, though - looks like there are 9 OPP tests

Oh, it is failing after the first test because I had configured it to do that. I just removed that config. FYI.

[mattlindsey]

Cool, although I see a typo in 'e2ehelper.rb' - equire should be require. And I'm still not sure why OllamaProxy doesn't run with the tests, but maybe it' something on my end.

I might try working on one of your 'help wanted' tasks for fun. I'll let you know.

[nickthecook]

Ah, thanks. Fixed.

When you run ops server, do you see lines that start with opp? Do you have port 11337 open when the server is running? (Check with netstat -an | grep LISTEN | grep 11337 on Mac, netstat ltpn | grep 1137 on Linux, unless you have a preferred way.)

Good stuff - lmk if you have any questions!

[mattlindsey]

opp runs fine using ops server, but doesn't appear to be working on my mac using ops e2e since I get the 'Connection refused` error above, but it's not important at the moment. Everything else looks great.