ruby-concurrency / concurrent-ruby

Modern concurrency tools including agents, futures, promises, thread pools, supervisors, and more. Inspired by Erlang, Clojure, Scala, Go, Java, JavaScript, and classic concurrency patterns.
https://ruby-concurrency.github.io/concurrent-ruby/
Other
5.68k stars 418 forks source link

why use wait_for_termination method will stuck the code #1004

Open banzhihang1 opened 1 year ago

banzhihang1 commented 1 year ago
        user_ids,org,delete_restore = get_org_and_user(delete_restore_id)
        return unless user_ids.present?
        work_num = [user_ids.length,50].min
        thread_pool = Concurrent::FixedThreadPool.new(work_num)
        error_map = Concurrent::Map.new
        redis_key = RedisClient::DDRS_BULK_DELETE_KEY % org.id  ## TODO
        set_redis_total_num(redis_key,user_ids.length)

        user_ids.each do |id|
          thread_pool.post do
            #delete_single_user(id,error_map,redis_key)
            target_class = ClassFactory.create_dynamic_model("zendmodo", "privacy_settings", true)
            puts id
            puts target_class
          end
        end

       thread_pool.shutdown
       thread_pool.wait_for_termination

the wait_for_termination method will stuck the following code

target_class = ClassFactory.create_dynamic_model("zendmodo", "privacy_settings", true)
bensheldon commented 1 year ago

@banzhihang1 can you share the implementation of ClassFactory.create_dynamic_model? I'm imagining that its likely to be the problem here (i.e. not thread safe).

It would be most helpful if you can share a self-contained reproduction script without private/external dependencies.

banzhihang1 commented 1 year ago

@bensheldon this is code

class ClassFactory
  def self.create_dynamic_model(database_name, table_name, deletable=false)
    new_class = self.name + database_name.camelize + table_name.singularize.camelize

    if deletable
      existing_classes = ActiveRecord::Base.descendants.reject(&:abstract_class).select{ |m| m.table_name == table_name && m.database_name == database_name }

      if existing_classes.length == 1
        include_delete_restore(existing_classes[0])
        return existing_classes[0]
      end

      if existing_classes.length > 1
        primary = find_primary_existing_class(existing_classes)
        if primary.present?
          include_delete_restore(primary)
          return primary
        end
      end
    end

    existing_class = begin
      k = Module.const_get(new_class)
      if k.database_name == database_name && k.table_name == table_name
        k
      end
    rescue NameError
      nil
    end

    if existing_class
      include_delete_restore(existing_class) if deletable
      return existing_class
    end

    Object.const_set(new_class, Class.new(database_name == 'one_eye' ? ActiveRecord::Base : "#{database_name.camelcase}DatabaseRecord".constantize))
    klass = new_class.constantize
    klass.table_name = table_name
    klass.inheritance_column = nil

    if klass.primary_key == nil
      # Rails can't handle composite PKs.
      # We have to retrieve from the schema cache and explicitly set
      key = klass.connection.schema_cache.primary_keys(table_name)
      klass.primary_key = key
    end

    include_delete_restore(klass) if deletable
    klass
  end

  private

  def self.include_delete_restore(klass)
    klass.include(DeleteRestore::Deletable)
    klass.include(DeleteRestore::Restorable)
  end

  def self.find_primary_existing_class(existing_classes)
    # First check to see if any have been marked as primary
    primary = existing_classes.select { |c| c.instance_variable_defined?(:@primary_model_for_table) && c.instance_variable_get(:@primary_model_for_table) }
    return primary[0] if primary.length == 1

    # Then check to see if only one implements Deletable
    deletable = existing_classes.select { |c| c < DeleteRestore::Deletable }
    return deletable[0] if deletable.length == 1

    # Check to see if only one override the delete_batch method
    overriding = existing_classes.select { |c| c.respond_to?(:delete_batch) && c.method(:delete_batch).owner != DeleteRestore::Deletable::ClassMethods } 
    return overriding[0] if overriding.length == 1

    # If we can't narrow it down, return nil
    return nil
  end
end

The code did not run in, but got stuck in an external call