lorennorman / ruby-on-lambda

Automation scripts to help run Ruby 2.1 programs on AWS Lambda
MIT License
143 stars 27 forks source link

app.rb code not finding my required files #9

Open davidlbean opened 7 years ago

davidlbean commented 7 years ago

Hmmm...I refactored my largish app.rb into several files. At the top of my app.rb file, I try to require these other files, and ruby can't find them. I've tried

require './my_class.rb'

and

require_relative 'my_class.rb'

but neither seem to work. Here's what I get in the log files:

2017-01-21T00:28:06.308Z 7720d3b8-df70-11e6-89f4-35cd4eb32432 stderr:
: cannot load such file -- /var/task/lib/app/my_class.rb (LoadError)
from /var/task/lib/app/app.rb:5:in `<main>'

What's the proper way to reference my own code files from app.rb? Apologies if I'm missing something blatantly obvious to everyone else.

lorennorman: btw, thanks for creating this little package...it's super! I was forcing myself to jump through all sorts of node.js hoops and it was driving me crazy. I'm so much happier in the Ruby world.

foggy1 commented 7 years ago

Having a similar issue but with trying to require nokogiri. It seems to be compiling the extensions right and entering nokogiri.bundle into nokogiri/nokogiri, but when nokogiri rb comes to require 'nokogiri/nokogiri' I get

/var/task/lib/vendor/ruby/2.1.0/gems/nokogiri-1.6.6.2/lib/nokogiri.rb:29:inrequire': cannot load such file -- nokogiri/nokogiri`

tmhammer commented 7 years ago

I ran into the same issue using require, but using require_relative worked for me.

However I am also having the same issue loading Nokogiri.

lorennorman commented 7 years ago

Nokogiri is a native gem, which I mention being problematic in the README.

Think about it: you bundle everything up locally (OSX? Ubuntu? Windows?) and send it off to be executed elsewhere (Amazon Linux, specifically.) Well, a gem that has been compiled against OSX isn't going to work in Amazon Linux. You'd need to compile that one in an Amazon Linux instance. Which is probably possible? But certainly not something I bit off in this initial proof-of-concept. (But certainly something I'd like to see done!)

Please do investigate.

foggy1 commented 7 years ago

Thanks for the reply, @lorennorman. What impressed me was that nokogiri actually did appear to be building off of the extensions and putting things in the correct place; but, there was obviously no substitute for the native stuff that's required (I think on mac it needs to run against some xcode stuff).

lorennorman commented 7 years ago

No worries!

Yeah this stuff can be kind of fraught with confusion, but I'm seeing other tools appear that are improving the experience. I'm looking at ClaudiaJS https://claudiajs.com/ for a thing I'm working on right now at work. Perhaps I'll bring some of the learnings from it back here and make something similar for Ruby soon.

fagiani commented 7 years ago

@lorennorman @foggy1 @tmhammer I'd like to report here that usingrequire_relative has actually resolved for project dependencies and creating a EC2 box with Amazon Linux got me nokogiri working fine on Lambda but I'm facing a weird one with activerecord and pg gems. I'm posting the logs of a test here in the hopes someone has been there before and could help:

START RequestId: feab102b-5da5-11e7-bf18-739c73eb4ef0 Version: $LATEST
2017-06-30T15:08:39.144Z    feab102b-5da5-11e7-bf18-739c73eb4ef0    Starting process: ./app
2017-06-30T15:08:51.403Z    feab102b-5da5-11e7-bf18-739c73eb4ef0    stderr:
/var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
2017-06-30T15:08:51.403Z    feab102b-5da5-11e7-bf18-739c73eb4ef0    stderr:
: Could not load 'active_record/connection_adapters/postgresql_adapter'. Make sure that the adapter in config/database.yml is valid. If you use an adapter other than 'mysql', 'mysql2', 'postgresql' or 'sqlite3' add the necessary adapter gem to the Gemfile. (LoadError)
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `block in require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:240:in `load_dependency'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/pg-0.21.0/lib/pg.rb:4:in `<top (required)>'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `block in require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:240:in `load_dependency'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/postgresql_adapter.rb:17:in `<top (required)>'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `block in require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:240:in `load_dependency'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/connection_specification.rb:175:in `spec'
from /var/task/lib/vendor/ruby/2.1.0/gems/activerecord-4.2.9/lib/active_record/connection_handling.rb:50:in `establish_connection'
from /var/task/lib/app/app.rb:12:in `<main>'

2017-06-30T15:08:51.744Z    feab102b-5da5-11e7-bf18-739c73eb4ef0    {"errorMessage":"Process \"./app\" exited with code: 1"}
END RequestId: feab102b-5da5-11e7-bf18-739c73eb4ef0
fagiani commented 7 years ago

I've found the error above to be misleading as activerecord will rescue and customize the error. The actual exception is linked to the pg_ext.so which does exists on the absolute path listed but I'm guessing there is some kind of missing native libraries would be the real error reason which I've been unable to figure out so far.

START RequestId: a9e88496-6005-11e7-924c-e7c37f2df3e0 Version: $LATEST
2017-07-03T15:38:30.977Z    a9e88496-6005-11e7-924c-e7c37f2df3e0    Starting process: ./app
2017-07-03T15:38:42.337Z    a9e88496-6005-11e7-924c-e7c37f2df3e0    stderr:
/var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
2017-07-03T15:38:42.456Z    a9e88496-6005-11e7-924c-e7c37f2df3e0    stderr:
: libpq.so.5: cannot open shared object file: No such file or directory - /var/task/lib/vendor/ruby/2.1.0/extensions/x86_64-linux/2.1.0-static/pg-0.21.0/pg_ext.so - libpq.so.5: cannot open shared object file: No such file or directory - /var/task/lib/vendor/ruby/2.1.0/extensions/x86_64-linux/2.1.0-static/pg-0.21.0/pg_ext.so - Could not load 'active_record/connection_adapters/postgresql_adapter'. Make sure that the adapter in config/database.yml is valid. If you use an adapter other than 'mysql', 'mysql2', 'postgresql' or 'sqlite3' add the necessary adapter gem to the Gemfile. (LoadError)
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `block in require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:240:in `load_dependency'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/pg-0.21.0/lib/pg.rb:4:in `<top (required)>'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `block in require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:240:in `load_dependency'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/postgresql_adapter.rb:17:in `<top (required)>'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `block in require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:240:in `load_dependency'
from /var/task/lib/vendor/ruby/2.1.0/gems/activesupport-4.2.9/lib/active_support/dependencies.rb:274:in `require'
from /var/task/lib/vendor/ruby/2.1.0/gems/activerecord-4.2.9/lib/active_record/connection_adapters/connection_specification.rb:175:in `spec'
from /var/task/lib/vendor/ruby/2.1.0/gems/activerecord-4.2.9/lib/active_record/connection_handling.rb:50:in `establish_connection'
from /var/task/lib/app/app.rb:12:in `<main>'

2017-07-03T15:38:42.676Z    a9e88496-6005-11e7-924c-e7c37f2df3e0    {"errorMessage":"Process \"./app\" exited with code: 1"}
END RequestId: a9e88496-6005-11e7-924c-e7c37f2df3e0

I'd appreciate if anyone finds any obvious hits that I might be missing on those lines.

fagiani commented 7 years ago

I've just figured the resolution for this issue. Posted the answer here for the record if anybody else gets trapped on this.

Hope that helps!

lorennorman commented 7 years ago

Nice work getting it fixed and following up with your progress, thank you!

On Mon, Jul 3, 2017 at 5:38 PM Paulo Fagiani notifications@github.com wrote:

I've just figured the resolution for this issue. Posted the answer here https://stackoverflow.com/a/44894150/1463744 for the record if anybody else gets trapped on this.

Hope that helps!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lorennorman/ruby-on-lambda/issues/9#issuecomment-312738042, or mute the thread https://github.com/notifications/unsubscribe-auth/AABFIUlRmPNcoAi8MFL4UoBDdD3BYr-dks5sKV9JgaJpZM4Lp5HS .