FahadNoor / py_crunchbase

A python client for Crunchbase's REST API
MIT License
35 stars 3 forks source link

Issue in after_id being automatically called #12

Open yash85763 opened 3 months ago

yash85763 commented 3 months ago

Hi @FahadNoor I was trying to get the jobs data of an organization, and in the params dictionary it is automatically sending after_id, even when the limit is 2. Everytime limit 2 and updated after_id is being sent to crunchbase, this leads to continuous data pull. I tried to create custom Pagination and and updated the query builder a little bit to only send limit not after_id along with it. But could you please explain what can be a solution?

right now my custom pagination code is below:

  class CustomPaginated(Paginated):
      def __init__(self, api_client, entity_id, card_id, card_field_ids, limit):
          super().__init__()
          self.api_client = api_client
          self.entity_id = entity_id
          self.card_id = card_id
          self.card_field_ids = card_field_ids
          self.limit = limit
          self.next_id = None
          self.previous_id = None
          self.total_results = 0

      def set_next(self, current_list):
          pass

      def set_previous(self, current_list):
          pass

      def execute(self) -> list:
          if self.total_results >= self.limit:
              return []

          params = {
              'limit': self.limit,
              'card_field_ids': ','.join(self.card_field_ids)
          }
          response = self.api_client.request(self.entity_id, self.card_id, params)
          data = response.get('data', [])
          self.total_results += len(data)

          if self.total_results >= self.limit:
              return data[:self.limit - self.total_results]

          return data

      def reset_pagination(self):
          self.next_id = None
          self.previous_id = None
          self.total_results = 0
FahadNoor commented 3 months ago

Hi @yash85763, can you please share the code so that I can reproduce the issue? I am not clear on what exactly is the issue you are facing.

yash85763 commented 3 months ago
card_fields_ids = ['is_current', 'person_identifier', 'identifier', 'short_description', 
                      'organization_identifier', 'created_at', 'started_on', 'ended_on',  
                      'employee_featured_order', 'title', 'updated_at', 'job_type']
# Access the organizations API
org_api = pycb.organizations_api()

# Define the cards for organization jobs
cards = Cards.Organization

call_limit = 1

# Fetch the organization entity data
entity = org_api.get_cards(
    entity_id=Company_2_look_4,
    card_id= cards.jobs, 
    card_field_ids=card_fields_ids,
    limit=call_limit
)

output:

With params: {'limit': 1, 'card_field_ids': 'ended_on,title,short_description,organization_identifier,started_on,employee_featured_order,person_identifier,created_at,job_type,updated_at,is_current,identifier'} Response Status Code: 200

and the output continues with updated next_id and previous_id