googleapis / google-cloud-ruby

Google Cloud Client Library for Ruby
https://googleapis.github.io/google-cloud-ruby/
Apache License 2.0
1.35k stars 545 forks source link

Thread error in fork() with Rails on MacOS #4455

Closed rept closed 4 years ago

rept commented 4 years ago

Running MacOS Mojave.

When I run the example explained here:

https://cloud.google.com/text-to-speech/docs/reference/libraries#client-libraries-install-ruby

I get this error:

objc[80556]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. objc[80556]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

On this line:

response = client.synthesize_speech synthesis_input, voice, audio_config

Console exits... I've already tried exporting this:

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

But that doesn't seem to make any difference.

blowmage commented 4 years ago

What Ruby version are you using?

Are you calling fork in your code anywhere?

quartzmo commented 4 years ago

I could not reproduce this on Mac OS Catalina 10.15.1 using ruby 2.4.5p335 (2018-10-18 revision 65137) [x86_64-darwin17]. The example runs correctly for me.

blowmage commented 4 years ago

What version of the google-cloud-speech, grpc, and google-protobuf gems are you using?

Can you post a minimal snippet of code that produces this error so we can try to reproduce?

quartzmo commented 4 years ago

Here's the stack that works for me with the example linked in the issue description:

$ bundle
Using rake 12.3.3
Using public_suffix 4.0.1
Using addressable 2.7.0
Using ast 2.4.0
Using bundler 1.17.1
Using docile 1.3.2
Using multipart-post 2.1.1
Using faraday 0.17.0
Using google-protobuf 3.10.1 (universal-darwin)
Using googleapis-common-protos-types 1.0.4
Using grpc 1.25.0 (universal-darwin)
Using googleapis-common-protos 1.3.9
Using jwt 2.2.1
Using memoist 0.16.1
Using multi_json 1.14.1
Using os 1.0.1
Using signet 0.12.0
Using googleauth 0.10.0
Using rly 0.2.3
Using google-gax 1.8.1
Using google-cloud-text_to_speech 0.5.1 from source at `.`
Using jaro_winkler 1.5.4
Using parallel 1.19.1
Using parser 2.6.5.0
Using rainbow 3.0.0
Using ruby-progressbar 1.10.1
Using unicode-display_width 1.6.0
Using rubocop 0.74.0
Using google-style 1.24.0
Using json 2.2.0
Using minitest 5.11.3
Using redcarpet 3.5.0
Using simplecov-html 0.10.2
Using simplecov 0.17.1
Using yard 0.9.20
rept commented 4 years ago

What Ruby version are you using?

Are you calling fork in your code anywhere?

ruby '2.6.3'

No I'm not calling fork. The code is on my laptop, I'll answer your other questions asap. Thanks.

rept commented 4 years ago

What version of the google-cloud-speech, grpc, and google-protobuf gems are you using?

google-cloud-text_to_speech (0.5.1)
grpc (1.25.0)
google-protobuf (3.9.1)
rept commented 4 years ago

This is the bundle output.

Using rake 12.3.2
Using concurrent-ruby 1.0.5
Using i18n 1.1.0
Using minitest 5.11.3
Using thread_safe 0.3.6
Using tzinfo 1.2.5
Using activesupport 5.2.1
Using builder 3.2.3
Using erubi 1.7.1
Using mini_portile2 2.3.0
Using nokogiri 1.8.5
Using rails-dom-testing 2.0.3
Using crass 1.0.4
Using loofah 2.2.2
Using rails-html-sanitizer 1.0.4
Using actionview 5.2.1
Using rack 2.0.5
Using rack-test 1.1.0
Using actionpack 5.2.1
Using nio4r 2.3.1
Using websocket-extensions 0.1.3
Using websocket-driver 0.7.0
Using actioncable 5.2.1
Using globalid 0.4.1
Using activejob 5.2.1
Using mini_mime 1.0.1
Using mail 2.7.0
Using actionmailer 5.2.1
Using arbre 1.1.1
Using coffee-script-source 1.12.2
Using execjs 2.7.0
Using coffee-script 2.4.1
Using method_source 0.9.0
Using thor 0.20.0
Using railties 5.2.1
Using coffee-rails 4.2.2
Using formtastic 3.1.5
Using formtastic_i18n 0.6.0
Using has_scope 0.7.2
Using responders 2.4.0
Using inherited_resources 1.9.0
Using jquery-rails 4.3.3
Using kaminari-core 1.1.1
Using kaminari-actionview 1.1.1
Using activemodel 5.2.1
Using arel 9.0.0
Using activerecord 5.2.1
Using kaminari-activerecord 1.1.1
Using kaminari 1.1.1
Using ransack 2.1.1
Using rb-fsevent 0.10.3
Using ffi 1.9.25
Using rb-inotify 0.9.10
Using sass-listen 4.0.0
Using sass 3.6.0
Using sprockets 3.7.2
Using activeadmin 1.4.3
Using multi_json 1.13.1
Using activerecord-session_store 1.1.1
Using mimemagic 0.3.2
Using marcel 0.3.3
Using activestorage 5.2.1
Using public_suffix 3.0.3
Using addressable 2.5.2
Using amq-protocol 2.3.0
Using bcrypt 3.1.12
Using coderay 1.1.2
Using better_errors 2.5.0
Using bindex 0.5.0
Using debug_inspector 0.0.3
Using binding_of_caller 0.8.0
Using msgpack 1.2.4
Using bootsnap 1.3.2
Using browser 2.5.3
Using bugsnag 6.11.1
Using bundler 1.17.2
Using bunny 2.13.0
Using byebug 10.0.2
Using xpath 3.1.0
Using capybara 3.9.0
Using capybara-email 3.0.1
Using launchy 2.4.3
Using capybara-screenshot 1.0.22
Using timers 4.3.0
Using celluloid-essentials 0.20.5
Using celluloid-extras 0.20.5
Using celluloid-fsm 0.20.5
Using celluloid-pool 0.20.5
Using celluloid-supervision 0.20.6
Using celluloid 0.17.4
Using celluloid-io 0.17.3
Using chartkick 3.2.1
Using childprocess 0.9.0
Using chronic 0.10.2
Using chunky_png 1.3.11
Using i18n_data 0.8.0
Using sixarm_ruby_unaccent 1.2.0
Using unicode_utils 1.4.0
Using countries 3.0.0
Using sort_alphabetical 1.1.0
Using country_select 4.0.0
Using declarative 0.0.10
Using declarative-option 0.1.0
Using orm_adapter 0.5.0
Using warden 1.2.7
Using devise 4.5.0
Using devise-i18n 1.6.5
Using diff-lcs 1.3
Using digest-crc 0.4.1
Using docile 1.3.2
Using unf_ext 0.0.7.5
Using unf 0.1.4
Using domain_name 0.5.20180417
Using erubis 2.7.0
Using kostya-sigar 2.0.6
Using state_machines 0.5.0
Using eye 0.10.0
Using eye-rotate 0.1.3
Using factory_bot 5.0.2
Using factory_bot_rails 5.0.1
Using faker 1.9.3
Using multipart-post 2.0.0
Using faraday 0.15.3
Using geocoder 1.5.2
Using jwt 2.2.1
Using memoist 0.16.1
Using os 1.0.1
Using signet 0.12.0
Using googleauth 0.10.0
Using httpclient 2.8.3
Using uber 0.1.0
Using representable 3.0.4
Using retriable 3.1.2
Using google-api-client 0.34.1
Using google-cloud-env 1.3.0
Using google-cloud-core 1.4.1
Using google-cloud-storage 1.24.0
Using google-protobuf 3.9.1 (universal-darwin)
Using googleapis-common-protos-types 1.0.4
Using grpc 1.25.0 (universal-darwin)
Using googleapis-common-protos 1.3.9
Using rly 0.2.3
Using google-gax 1.8.1
Using google-cloud-text_to_speech 0.5.1
Using groupdate 4.1.2
Using temple 0.8.0
Using tilt 2.0.8
Using haml 5.0.4
Using sexp_processor 4.11.0
Using ruby_parser 3.11.0
Using html2haml 2.2.0
Using haml-rails 1.0.0
Using hkdf 0.3.0
Using hstore_translate 2.0.0
Using http-cookie 1.0.3
Using jbuilder 2.7.0
Using json 2.1.0
Using json-schema 2.8.1
Using letter_opener 1.6.0
Using ruby_dep 1.5.0
Using listen 3.1.5
Using mime-types-data 3.2018.0812
Using mime-types 3.2.2
Using netrc 0.11.0
Using rest-client 2.0.2
Using mailgun-ruby 1.1.11
Using open4 1.3.4
Using mina 0.3.8
Using mina-multistage 1.0.2
Using mina-tail 0.0.2
Using newrelic_rpm 6.2.0.354
Using pg 1.1.3
Using phony 2.16.9
Using phony_rails 0.14.11
Using puma 3.12.0
Using sprockets-rails 3.2.1
Using rails 5.2.1
Using rails-observers 0.1.5
Using redis 4.0.3
Using rqrcode 0.10.1 from https://github.com/rept/rqrcode.git (at master@f5ebfd2)
Using rspec-support 3.8.0
Using rspec-core 3.8.0
Using rspec-expectations 3.8.2
Using rspec-mocks 3.8.0
Using rspec-rails 3.8.2
Using rubyzip 1.2.2
Using sass-rails 5.0.7
Using selenium-webdriver 3.14.1
Using sigdump 0.2.4
Using serverengine 2.0.7
Using serviceworker-rails 0.5.5
Using shoulda-matchers 4.0.1
Using simple_form 4.0.1
Using simplecov-html 0.10.2
Using simplecov 0.17.0
Using sneakers 2.11.0
Using spring 2.0.2
Using spring-watcher-listen 2.0.1
Using sqlite3 1.3.13
Using sweetalert2 7.25.6
Using timecop 0.9.1
Using turbolinks-source 5.2.0
Using turbolinks 5.2.0
Using twilio-ruby 5.15.2
Using uglifier 4.1.19
Using web-console 3.7.0
Using webdrivers 4.1.2
Using webpush 1.0.0
Using whenever 0.10.0
rept commented 4 years ago

When I start a new project with the same Gemfile and Gemfile.lock, it does work. I run the example in the link I posted and it doesn't give the

objc[8992]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.

error message. Both tested from console.

quartzmo commented 4 years ago

@rept If this error can't be reproduced, I think we should close this issue. Is that OK with you?

blowmage commented 4 years ago

Does the project that does have this behavior use vendored gems?

blowmage commented 4 years ago

Does bundle doctor give you anything interesting?

blowmage commented 4 years ago

@rept Are you using spring in your rails app?

Rosie-Brigham commented 4 years ago

I am also getting this error, whilst using Google Cloud Vision Image Annotator

image_annotator = ::Google::Cloud::Vision::ImageAnnotator.new

returns:

objc[75347]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

I also already tried exporting this: export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

but this makes no difference. I am also not calling fork() anywhere

Any help much appreciated!!

Ruby version: 2.6.3 rails: 5.2.3 My mac is running Mojave 10.14.6

Bundle output:

Using rake 12.3.3
Using concurrent-ruby 1.1.5
Using i18n 1.7.0
Using minitest 5.12.2
Using thread_safe 0.3.6
Using tzinfo 1.2.5
Using activesupport 5.2.3
Using builder 3.2.3
Using erubi 1.9.0
Using mini_portile2 2.4.0
Using nokogiri 1.10.4
Using rails-dom-testing 2.0.3
Using crass 1.0.5
Using loofah 2.3.1
Using rails-html-sanitizer 1.3.0
Using actionview 5.2.3
Using rack 2.0.7
Using rack-test 1.1.0
Using actionpack 5.2.3
Using nio4r 2.3.1
Using websocket-extensions 0.1.3
Using websocket-driver 0.7.0
Using actioncable 5.2.3
Using globalid 0.4.2
Using activejob 5.2.3
Using mini_mime 1.0.1
Using mail 2.7.1
Using actionmailer 5.2.3
Using activemodel 5.2.3
Using case_transform 0.2
Using jsonapi-renderer 0.2.2
Using active_model_serializers 0.10.10
Using arel 9.0.0
Using activerecord 5.2.3
Using mimemagic 0.3.3
Using marcel 0.3.3
Using activestorage 5.2.3
Using public_suffix 3.1.0
Using addressable 2.6.0
Using api-pagination 4.8.2
Using aws-eventstream 1.0.3
Using aws-partitions 1.168.0
Using aws-sigv4 1.1.0
Using jmespath 1.4.0
Using aws-sdk-core 3.54.0
Using aws-sdk-kms 1.21.0
Using aws-sdk-s3 1.40.0
Using bcrypt 3.1.13
Using bindex 0.7.0
Using buftok 0.2.0
Using bundler 1.17.3
Using byebug 11.0.1
Using childprocess 1.0.1
Using climate_control 0.2.0
Using coderay 1.1.2
Using coffee-script-source 1.12.2
Using execjs 2.7.0
Using coffee-script 2.4.1
Using method_source 0.9.2
Using thor 0.20.3
Using railties 5.2.3
Using coffee-rails 4.2.2
Using connection_pool 2.2.2
Using database_cleaner 1.7.0
Using orm_adapter 0.5.0
Using responders 3.0.0
Using warden 1.2.8
Using devise 4.7.1
Using diff-lcs 1.3
Using unf_ext 0.0.7.6
Using unf 0.1.4
Using domain_name 0.5.20180417
Using dotenv 2.7.2
Using dotenv-rails 2.7.2
Using equalizer 0.0.11
Using factory_bot 4.11.1
Using factory_bot_rails 4.11.1
Using multipart-post 2.1.1
Using faraday 0.17.0
Using ffi 1.11.1
Using font-awesome-rails 4.7.0.5
Using google-protobuf 3.11.0 (universal-darwin)
Using googleapis-common-protos-types 1.0.4
Using grpc 1.25.0 (universal-darwin)
Using googleapis-common-protos 1.3.9
Using jwt 2.2.1
Using memoist 0.16.1
Using multi_json 1.13.1
Using os 1.0.1
Using signet 0.12.0
Using googleauth 0.10.0
Using rly 0.2.3
Using google-gax 1.8.1
Using google-cloud-vision 0.37.1
Using temple 0.8.1
Using tilt 2.0.9
Using haml 5.1.1
Using http-cookie 1.0.3
Using http-form_data 2.1.1
Using http_parser.rb 0.6.0
Using http 3.3.0
Using mime-types-data 3.2019.0331
Using mime-types 3.2.2
Using multi_xml 0.6.0
Using httparty 0.17.0
Using jbuilder 2.9.1
Using jquery-rails 4.3.3
Using kaminari-core 1.1.1
Using kaminari-actionview 1.1.1
Using kaminari-activerecord 1.1.1
Using kaminari 1.1.1
Using libv8 7.3.492.27.1 (x86_64-darwin-18)
Using rb-fsevent 0.10.3
Using rb-inotify 0.10.0
Using ruby_dep 1.5.0
Using listen 3.1.5
Using memoizable 0.4.2
Using mini_magick 4.9.5
Using mini_racer 0.2.6
Using naught 1.1.0
Using terrapin 0.6.0
Using paperclip 6.1.0
Using pg 0.21.0
Using pry 0.12.2
Using puma 3.12.1
Using rack-attack 6.1.0
Using rack-proxy 0.6.5
Using sprockets 3.7.2
Using sprockets-rails 3.2.1
Using rails 5.2.3
Using rails-controller-testing 1.0.4
Using rails_serve_static_assets 0.0.5
Using rails_stdout_logging 0.0.5
Using rails_12factor 0.0.3
Using rainbow 3.0.0
Using react_on_rails 11.2.1
Using rearmed 2.0.1
Using rspec-support 3.8.0
Using rspec-core 3.8.0
Using rspec-expectations 3.8.3
Using rspec-mocks 3.8.0
Using rspec 3.8.0
Using rspec-rails 3.8.2
Using rspec_junit_formatter 0.2.3
Using rubyzip 1.3.0
Using sass-listen 4.0.0
Using sass 3.7.4
Using sass-rails 5.0.7
Using selenium-webdriver 3.142.3
Using shoulda-context 1.2.2
Using shoulda-matchers 2.8.0
Using shoulda 3.5.0
Using simple_oauth 0.3.1
Using smart_listing 1.2.2
Using spring 2.0.2
Using spring-watcher-listen 2.0.1
Using turbolinks-source 5.2.0
Using turbolinks 5.2.0
Using twitter 6.2.0
Using uglifier 4.1.20
Using web-console 3.7.0
Using webpacker 3.6.0
Rosie-Brigham commented 4 years ago

In addition to ^ this is the script I'm trying to get working:

require "google/cloud/vision"

image_annotator = Google::Cloud::Vision::ImageAnnotator.new

detect_obj = image_annotator.object_localization_detection image: 'https://avatars2.githubusercontent.com/u/4891353?s=460&v=4'

detect_obj.responses.each do |response|
  response.localized_object_annotations.each do |object|
    puts "#{object.name} (confidence: #{object.score})"
  end
end

Is I run it as a plain ruby file, it works fine (ruby [file-directory]) - it's only when I run is as a rails files (rails runner [file-directory]) that it fail whales. Is this a problem with how rails is working?

I am a front end dev normally and all this is waaaaaayyyy over my head!

blowmage commented 4 years ago

@Rosie-Brigham If you are calling rails runner on an app that is using Spring, then that command forks the rails process before running the script. This error is raised when the GRPC gem has been instantiated on a different running process other than the current process. So it looks like your application is instantiating a GRPC client during application boot, which means every GRPC client is unable to be used in a forked process like the one rails runner creates. This is why calling the script directly does not raise the error.

blowmage commented 4 years ago

Possible dup of #4387.

Rosie-Brigham commented 4 years ago

Thanks @blowmage , that really helps - playing around more if I make it into a class and call it from the console it throws the error, but if I instead instantiate it within a controller it seems to run fine.

Do you know if there is a workaround where I can get this working within the context of a class that can be run from the console or runner as I intend to mainly use it in that way?

Made into basic class:

require "google/cloud/vision"

class ImageTagJob
  def perform
    detect_obj = image_annotator.label_detection image: 'https://avatars2.githubusercontent.com/u/4891353?s=460&v=4'

    detect_obj.responses.each do |response|
      response.localized_object_annotations.each do |object|
        puts "#{object.name} (confidence: #{object.score})"
      end
    end
  end

  private

  def image_annotator
    @image_annotator ||= Google::Cloud::Vision::ImageAnnotator.new
  end
end

Called in rails C: same error message

Put into a controller or similar,

class WelcomeController < ApplicationController
  def index
    ImageTagJob.new.perform
  end
end

works fine

rept commented 4 years ago

@blowmage No vendored gems. bundle doctor says everything is OK. I do have spring in my Gemfile. I commented this out did bundle install and now it's working!!! Thanks a lot for your assistance!

rept commented 4 years ago

@blowmage now I'm clueless. I've uncommented spring again, did bundle install and it keeps working. The only difference is that I've closed the original terminal window and am working in a new one. Could this have been something that got messed up in the terminal? (from which I run rails c)?

blowmage commented 4 years ago

Spring starts a process of your app, and then forks the process whenever you open a console or run a task against it. Each action is run on a forked process of the original process. So if your app is creating an client instance during boot, that client will not be able to be run on the forked process, and the forked process will be unable to create a new client instance.

dorianmariecom commented 4 years ago

I had the same issue with google translate api, solution was to remove spring from Gemfile

scarroll32 commented 3 years ago

You can also stop spring

bin/spring stop