scientist-softserv / louisville-hyku

Other
0 stars 0 forks source link

works with a filename that includes the letter “s” before the filetype are skipped #172

Closed crisr15 closed 1 year ago

crisr15 commented 1 year ago

Summary

During bulkrax ingest some children have alphanumeric filenames, and I’ve noticed that when a filename includes the letter “s” before the filetype it is skipped – is there something in the code that treats an “s” before a filetype as a stop word? See, for example, https://hyku-docker-dev.library.louisville.edu/concern/images/ulua_2006_003_008_299/ - it skips from ulua_2006_003_008_299r to ulua_2006_003_008_299t

Acceptance Criteria

Testing Instructions

Notes

When looking at the file mentioned above I noticed that the file ulua_2006_003_008_299f was also missing.
Confirmed in meeting that 's' is the repeated problem. Do not worry about f missing currently on this one item.

alishaevn commented 1 year ago

warriors_with_an_s.zip

I uploaded the file above and had no problems with any files being uploaded. 🤔 in the screenshot below, the works with 's added to the end of the title are the ones whose file names ended in XXXs.jpg

Image

alishaevn commented 1 year ago
stephs.jpg ![Image](https://user-images.githubusercontent.com/29032869/228576370-9cfbb3a0-3823-4ba5-8faa-32762e98d692.png)
klays.jpg ![Image](https://user-images.githubusercontent.com/29032869/228577000-fb9e0f7d-b629-4cd7-9566-b8e331b801fc.png)
drays.jpg ![Image](https://user-images.githubusercontent.com/29032869/228577321-3654702b-710f-4bb9-9870-660e408c3ab5.png)
alishaevn commented 1 year ago

celtics.csv

the test in my comment above was done by uploading a csv. I then imported the celtics.csv file above using the server path upload method. it also uploaded fine.

I am unable to reproduce this error locally so far.

Image

alishaevn commented 1 year ago

@rachelihoward can you upload theULUA 2006_003_008_064 "a-s" files that are related to the "owen_compoundObjects_sample.csv" file, into this folder of the google drive?

since I'm unable to reproduce the error locally with my own files, I'd like to see if I can reproduce the error with files you all are using.

also, can you please take a look at my question to you and/or randy in the samvera channel?

alishaevn commented 1 year ago

by happenstance, I believe I've come across the actual issue. I think the problem is related to when the source_identifier ends in "s", not the file.

still troubleshooting.

rachelihoward commented 1 year ago

Alisha,

I just requested permissions for that drive.

Randy is working on setting up a user account – what were you planning to use it for?

Rachel

From: Alisha Evans @.> Sent: Wednesday, March 29, 2023 12:23 PM To: scientist-softserv/louisville-hyku @.> Cc: Howard, Rachel @.>; Mention @.> Subject: Re: [scientist-softserv/louisville-hyku] works with a filename that includes the letter “s” before the filetype are skipped (Issue #172)

You don't often get email from @.**@.>. Learn why this is importanthttps://aka.ms/LearnAboutSenderIdentification CAUTION: This email originated from outside of our organization. Do not click links, open attachments, or respond unless you recognize the sender's email address and know the contents are safe.

@rachelihowardhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Frachelihoward&data=05%7C01%7Crachel.howard%40louisville.edu%7Cdc803978e09a4923e16d08db3071d73f%7Cdd246e4a54344e158ae391ad9797b209%7C0%7C0%7C638157037771530527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=HfZnmYQVT27VRaZpquhKm%2BzQMfz0szqFNTt5gJOCmXg%3D&reserved=0 can you upload theULUA 2006_003_008_064 "a-s" files that are related to the "owen_compoundObjects_sample.csv" file, into this folderhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdrive.google.com%2Fdrive%2Ffolders%2F1q9d-C4Y34EqPB-HbywmCgIWCqFT8qTsr&data=05%7C01%7Crachel.howard%40louisville.edu%7Cdc803978e09a4923e16d08db3071d73f%7Cdd246e4a54344e158ae391ad9797b209%7C0%7C0%7C638157037771530527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=i%2Fdq1iLJjhF%2FthhsXxvsDapqgtJ4VmmWFmy9zCXh9js%3D&reserved=0 of the google drive?

since I'm unable to reproduce the error locally with my own files, I'd like to see if I can reproduce the error with files you all are using.

also, can you please take a look at my question to you and/or randy in the samvera channelhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsamvera.slack.com%2Farchives%2FC034S4UF53L%2Fp1680042780083409&data=05%7C01%7Crachel.howard%40louisville.edu%7Cdc803978e09a4923e16d08db3071d73f%7Cdd246e4a54344e158ae391ad9797b209%7C0%7C0%7C638157037771530527%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=SHBj8Qx5MWCJM9YFpPG5rfamTXJPcdkcD0zzW6eyv6A%3D&reserved=0?

— Reply to this email directly, view it on GitHubhttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fscientist-softserv%2Flouisville-hyku%2Fissues%2F172%23issuecomment-1488918323&data=05%7C01%7Crachel.howard%40louisville.edu%7Cdc803978e09a4923e16d08db3071d73f%7Cdd246e4a54344e158ae391ad9797b209%7C0%7C0%7C638157037771686765%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OElcCSIsYL1rPycdYx7luDaNPOPjGDIIJnSpsGuVXhE%3D&reserved=0, or unsubscribehttps://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FA5TKJLHAOP2R4XCEPREFMB3W6ROVVANCNFSM6AAAAAAUWUK2IU&data=05%7C01%7Crachel.howard%40louisville.edu%7Cdc803978e09a4923e16d08db3071d73f%7Cdd246e4a54344e158ae391ad9797b209%7C0%7C0%7C638157037771686765%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=6756MRVSRFzYmYYVD2npX%2FpVywN7wBCSazeiI%2BnDml8%3D&reserved=0. You are receiving this because you were mentioned.Message ID: @.**@.>>

rachelihoward commented 1 year ago

That's great news, Alisha!

rachelihoward commented 1 year ago

Sending files to the Google Drive now - it says it will take over an hour. Also, please remove ULUA 2006_003_008_230t from the csv. I don't know why I eliminated the parent and the rest of the children for that one from the sample set but it's not useful to just have that one child record. I think there are plenty of other examples without it!

alishaevn commented 1 year ago

kevons.zip

importer 7 ![Image](https://user-images.githubusercontent.com/29032869/228674554-188a98df-e9be-4083-9aa5-9fae5400a292.png)
i = Bulkrax::Importer.find(7)
e = i.entries.last

begin
  e.factory.run
rescue
  e
end

running the above in my rails console has led me to Hyrax::Actors::TransactionalRequest#create. this method returns false. according to the comment above the method, returning true means that the create was successful. so I think this is where the problem lies. not sure why though.

alishaevn commented 1 year ago

according to the console, next_actor inside transactional request should be Hyrax::Actors::OptimisticLockValidator. that class doesn't have a create method in hyrax 2.9.6 though. 🤔

alishaevn commented 1 year ago
actor stack ``` bash => #>>>>>>>>>>>>>>, @pdf_paths=[]>, @pdf_paths=[]>>> ```
alishaevn commented 1 year ago
actor stack methods ``` bash Hyrax::Actors::TransactionalRequest#create => false 12: def create(env) 13: next_actor.create(env) 14: end Hyrax::Actors::AbstractActor#create 91: delegate :create, to: :next_actor Hyrax::Actors::CreateWithRemoteFilesActor#create => false 18: def create(env) 19: remote_files = env.attributes.delete(:remote_files) 20: next_actor.create(env) && attach_files(env, remote_files) 21: end NewspaperWorks::Actors::NewspaperWorksUploadActor#create 4: def create(env) 5: # Ensure that work has title, set from form data if present 6: ensure_title(env) 7: # If NewspaperIssue, we might have a PDF to split; make a list of 8: # paths to PDF uploads before next_actor removes them 9: # from env.attributes, with state kept in instance variable 10: # until late in "heading back up" phase of the actor stack, where 11: # correct depositor value is already set on the issue (only 12: # at that point should we queue the job to create child pages). 13: @pdf_paths = [] 14: hold_upload_paths(env) if env.curation_concern.class == NewspaperIssue 15: # pass to next actor, then handle issue uploads after other actors 16: # that are lower on the stack 17: next_actor.create(env) && after_other_actors(env) 18: end Hyrax::Actors::CreateWithFilesActor#create => false 10: def create(env) => 11: uploaded_file_ids = filter_file_ids(env.attributes.delete(:uploaded_files)) 12: files = uploaded_files(uploaded_file_ids) 13: # OVERRIDE: Hyrax: Split PDF into jpg for each page and sent to attach files method 14: return false unless validate_files(files, env) 15: return false unless next_actor.create(env) 16: ConvertPdfToJpgJob.perform_later(files, env.curation_concern, env.attributes, env.user.id) if files.present? 17: # END OVERRIDE 18: attach_files(files, env) 19: end Hyrax::Actors::CollectionsMembershipActor#create 15: def create(env) => 16: extract_collection_id(env) 17: assign_nested_attributes_for_collection(env) && next_actor.create(env) 18: end Hyrax::Actors::AddToWorkActor#create 6: def create(env) => 7: work_ids = env.attributes.delete(:in_works_ids) 8: 9: can_edit_works?(env, work_ids) && next_actor.create(env) && add_to_works(env, work_ids) 10: end Hyrax::Actors::AbstractActor#create => 91: delegate :create, to: :next_actor # => Hyrax::Actors::AbstractActor#create => 91: delegate :create, to: :next_actor # => Hyrax::Actors::AbstractActor#create => 91: delegate :create, to: :next_actor Hyrax::Actors::DefaultAdminSetActor#create 12: def create(env) => 13: ensure_admin_set_attribute!(env) 14: next_actor.create(env) 15: end Hyrax::Actors::InterpretVisibilityActor#create 84: def create(env) => 85: intention = Intention.new(env.attributes) 86: attributes = intention.sanitize_params 87: new_env = Environment.new(env.curation_concern, env.current_ability, attributes) 88: validate(env, intention, attributes) && apply_visibility(new_env, intention) && 89: next_actor.create(new_env) 90: end Hyrax::Actors::TransferRequestActor#create 8: def create(env) => 9: next_actor.create(env) && create_proxy_deposit_request(env) 10: end Hyrax::Actors::ApplyPermissionTemplateActor#create 8: def create(env) => 9: add_edit_users(env) 10: next_actor.create(env) 11: end Hyrax::Actors::AbstractActor#create => 91: delegate :create, to: :next_actor # => Hyrax::Actors::AbstractActor#create => 91: delegate :create, to: :next_actor # => Hyrax::Actors::AbstractActor#create => 91: delegate :create, to: :next_actor Hyrax::Actors::ModelActor#create 16: def create(env) => 17: model_actor(env).create(env) 18: end Hyrax::Actors::BaseActor#create 18: def create(env) => 19: apply_creation_data_to_curation_concern(env) 20: apply_save_data_to_curation_concern(env) 21: save(env) && next_actor.create(env) && run_callbacks(:after_create_concern, env) 22: end ```

I stepped through each method in the actor stack, seen above, and the part that's failing is the rails #save method in BaseActor. according to apidock: "If perform_validation is true validations run. If any of them fail the action is cancelled and save returns false." I already knew that the validation was failing, but I assumed it was because the "s" was getting dropped somewhere prior to the save. however, per the below....the "s" is still present on the identifier property.

so far I'm unable to find where the validation is being set in the bulkrax or lv code.

# /usr/local/bundle/gems/hyrax-2.9.6/app/actors/hyrax/actors/base_actor.rb:68 Hyrax::Actors::BaseActor#save
67: def save(env)
68:   env.curation_concern.save
69: end

env.curation_concern.save => false

# note that the identifier is still plural...
[15] pry(#<Hyrax::Actors::TextActor>)> env.curation_concern
=> #<Text id: nil, head: [], tail: [], depositor: "admin@example.com", title: ["Kevon Looney"], date_uploaded: "2023-03-30 15:37:32", date_modified: "2023-03-30 15:37:34", state: nil, proxy_depositor: nil, on_behalf_of: nil, arkivo_checksum: nil, owner: nil, is_child: nil, slug: "kevons", alternative_title: [], collection_information: [], contributor_role: ["publisher"], creator_role: [], date_digital: nil, digitization_specification: "digitization_specification", format: "", location: [], media_type: ["Text"], mesh: [], ordering_information: nil, people_represented: [], resource_query: [], issue: "No. 2.", searchable_text: "Looney always holds us down!", table_of_contents: nil, volume: "Vol 2.", label: nil, relative_path: nil, import_url: nil, resource_type: ["Newspapers"], creator: [], contributor: ["Adam Silver"], description: ["Dub nation"], keyword: [], license: [], rights_statement: [], publisher: ["repository"], date_created: ["2/9/2022"], subject: ["This is the subject"], language: [], identifier: ["Kevons"], based_near: [], related_url: [], bibliographic_citation: [], source: ["The source"], access_control_id: nil, representative_id: nil, thumbnail_id: nil, rendering_ids: [], admin_set_id: "admin_set/default", embargo_id: nil, lease_id: nil>
alishaevn commented 1 year ago
[16] pry(#<Hyrax::Actors::TextActor>)> env.curation_concern.valid?
=> false
[17] pry(#<Hyrax::Actors::TextActor>)> env.curation_concern.errors.messages
=> {:identifier=>["must be unique"]}
[18] pry(#<Hyrax::Actors::TextActor>)> step

From: /usr/local/bundle/gems/active-fedora-12.2.3/lib/active_fedora/associations/builder/orders.rb:44 ActiveFedora::Associations::Builder::Orders::FixFirstLast#save:

    43: def save(*args)
 => 44:   super.tap do |result|
    45:     apply_first_and_last if result
    46:   end
    47: end

[18] pry(#<Text>)> step

From: /usr/local/bundle/gems/active-fedora-12.2.3/lib/active_fedora/validations.rb:50 ActiveFedora::Validations#save:

    49: def save(options = {})
 => 50:   perform_validations(options) ? super : false
    51: end

[18] pry(#<Text>)> perform_validations(options)
=> false
alishaevn commented 1 year ago

@bkiahstroud suggested I see if this same error happens when creating a work manually; it does. this confirms that the problem is somewhere in the hyrax pathway.

I created a work with a title and item number of "single". no file or any other metadata. I then tried creating a second work with a title and item number of "singles". it failed with the error below. I then created a third work with a title and item number of "doubles". that succeeded. so having an "s" at the end of the identifier alone isn't a problem. it's when that identifier without an "s" is the same as another work that already exists.

single ``` bash #>, proxy_depositor: nil, on_behalf_of: nil, arkivo_checksum: nil, owner: nil, is_child: nil, slug: "single", alternative_title: [], collection_information: [], contributor_role: [], creator_role: [], date_digital: nil, digitization_specification: nil, format: nil, location: [], media_type: [], mesh: [], ordering_information: nil, people_represented: [], resource_query: [], issue: nil, searchable_text: nil, table_of_contents: nil, volume: nil, label: nil, relative_path: nil, import_url: nil, resource_type: [], creator: [], contributor: [], description: [], keyword: [], license: [], rights_statement: [], publisher: [], date_created: [], subject: [], language: [], identifier: ["single"], based_near: [], related_url: [], bibliographic_citation: [], source: [], access_control_id: "1f472c1b-6603-46ec-8606-8e9a4e3a542b", representative_id: nil, thumbnail_id: nil, rendering_ids: [], admin_set_id: "admin_set/default", embargo_id: nil, lease_id: nil> ``` ![Image](https://user-images.githubusercontent.com/29032869/228978261-1b7be68e-2780-4164-ba2c-1165774f9531.png)
singles ![Image](https://user-images.githubusercontent.com/29032869/228976792-b9031ec4-e205-468c-b50b-0d87fa176cc7.png)
doubles ``` bash #>, proxy_depositor: nil, on_behalf_of: nil, arkivo_checksum: nil, owner: nil, is_child: nil, slug: "doubles", alternative_title: [], collection_information: [], contributor_role: [], creator_role: [], date_digital: nil, digitization_specification: nil, format: nil, location: [], media_type: [], mesh: [], ordering_information: nil, people_represented: [], resource_query: [], issue: nil, searchable_text: nil, table_of_contents: nil, volume: nil, label: nil, relative_path: nil, import_url: nil, resource_type: [], creator: [], contributor: [], description: [], keyword: [], license: [], rights_statement: [], publisher: [], date_created: [], subject: [], language: [], identifier: ["doubles"], based_near: [], related_url: [], bibliographic_citation: [], source: [], access_control_id: "ba09fc01-0d98-4177-b4ec-2157dcd9cbe3", representative_id: nil, thumbnail_id: nil, rendering_ids: [], admin_set_id: "admin_set/default", embargo_id: nil, lease_id: nil> ``` ![Image](https://user-images.githubusercontent.com/29032869/228978237-9dd65b9a-c9d3-4c03-937d-1ef9bc941d00.png)

this already appeared to be true because identifier is a property in Hyrax::BasicMetadata. it's created by rdf-vocab, but I don't see any validation rules yet.

dublin core specs

alishaevn commented 1 year ago

I went back through the actor stack and discovered that the identifier gets set somewhere in Hyrax::Actors::BaseActors#create. trying to save the new curation_concern, fails.

related console logs ``` bash # entering the method From: /usr/local/bundle/gems/hyrax-2.9.6/app/actors/hyrax/actors/base_actor.rb:19 Hyrax::Actors::BaseActor#create: 18: def create(env) => 19: apply_creation_data_to_curation_concern(env) 20: apply_save_data_to_curation_concern(env) 21: save(env) && next_actor.create(env) && run_callbacks(:after_create_concern, env) 22: end [14] pry(#)> env.curation_concern.identifier => [] # exiting the method From: /usr/local/bundle/gems/hyrax-2.9.6/app/actors/hyrax/actors/base_actor.rb:68 Hyrax::Actors::BaseActor#save: 67: def save(env) => 68: env.curation_concern.save 69: end [15] pry(#)> env.curation_concern.identifier => ["Kevons"] [16] pry(#)> env.curation_concern.save => false ```
alishaevn commented 1 year ago

further down the rabbit hole...the problem more closely lies somewhere in the ActiveFedora::Validations#run_validations! method calls.

when I entered ActiveFEDORA::Validations#valid?, it called super and led me to ActiveMODEL::Validations#valid?. once there, the errors object had empty hashes for the details and messages values. I stepped into the next methods, which was a series of callbacks kicked off by run_validations!(in ActiveFEDORA), and still didn't see anywhere where an error was set. however, as ActiveMODEL::Validations#valid? wrapped up and brought me back to ActiveFEDORA::Validations#valid?, errors.details and errors.messages had data.

errors ``` bash From: /usr/local/bundle/gems/activemodel-5.2.7.1/lib/active_model/validations.rb:338 ActiveModel::Validations#valid?: 336: def valid?(context = nil) 337: current_context, self.validation_context = validation_context, context => 338: errors.clear 339: run_validations! 340: ensure 341: self.validation_context = current_context 342: end [5] pry(#)> current_context => nil [6] pry(#)> self.validation_context => :create [7] pry(#)> errors => #, @details={}, @messages={}> From: /usr/local/bundle/gems/active-fedora-12.2.3/lib/active_fedora/validations.rb:70 ActiveFedora::Validations#valid?: 67: def valid?(context = nil) 68: context ||= default_validation_context 69: output = super(context) => 70: errors.empty? && output 71: end [27] pry(#)> output => false [28] pry(#)> errors => #, @details={:identifier=>[{:error=>"must be unique"}, {:error=>"must be unique"}, {:error=>"must be unique"}]}, @messages={:identifier=>["must be unique", "must be unique", "must be unique"]}> ```
alishaevn commented 1 year ago

further investigation has determined that line 132 of ActiveSupport::Callbacks#run_callbacks(v5.2.7.1) is where the error is being set. I'm unsure though what block was given and where. also, if a block was given, why did it yield false?

logs for ActiveSupport::Callbacks#run_callbacks ``` bash From: /usr/local/bundle/gems/activesupport-5.2.7.1/lib/active_support/callbacks.rb:132 ActiveSupport::Callbacks#run_callbacks: 127: end 128: 129: # Common case: no 'around' callbacks defined 130: if next_sequence.final? 131: next_sequence.invoke_before(env) => 132: env.value = !env.halted && (!block_given? || yield) 133: next_sequence.invoke_after(env) 134: env.value 135: else 136: invoke_sequence.call 137: end [12] pry(#)> self.errors => #, @details={}, @messages={}> [13] pry(#)> next From: /usr/local/bundle/gems/activesupport-5.2.7.1/lib/active_support/callbacks.rb:133 ActiveSupport::Callbacks#run_callbacks: 128: 129: # Common case: no 'around' callbacks defined 130: if next_sequence.final? 131: next_sequence.invoke_before(env) 132: env.value = !env.halted && (!block_given? || yield) => 133: next_sequence.invoke_after(env) 134: env.value 135: else 136: invoke_sequence.call 137: end 138: end [13] pry(#)> self.errors => #, @details={:identifier=>[{:error=>"must be unique"}]}, @messages={:identifier=>["must be unique"]}> [14] pry(#)> env.value => false [15] pry(#)> env => #, halted=false, value=false> [16] pry(#)> block_given? => true [17] pry(#)> yield => false ```
alishaevn commented 1 year ago

I looked at this ticket with jeremy and we found the problem. CustomSlugs::SlugBehavior#check_slug (in the lv codebase) was returning the error. that file is included in each of the model and collection classes. the method in question, is run as a validation for slugs.

we updated line 36 of the above method from possible_duplicates.count.positive? to possible_duplicates.any? { |pd| pd.slug == slug }. the result can be seen below.

checking for duplicates ``` bash # this is defined in [CustomSlugs::SlugBehavior#set_slug](https://github.com/scientist-softserv/louisville-hyku/blob/main/app/models/custom_slugs/slug_behavior.rb#L43-L46) irb(main):008:0> slug = 'singles'.truncate(75, omission: '').parameterize.underscore => "singles" irb(main):009:0> possible_duplicates = ActiveFedora::Base.where(slug_tesim: slug) => [# possible_duplicates.count.positive? => true irb(main):011:0> possible_duplicates => [#>, proxy_depositor: nil, on_behalf_of: nil, arkivo_checksum: nil, owner: nil, is_child: nil, slug: "single", alternative_title: [], collection_information: [], contributor_role: [], creator_role: [], date_digital: nil, digitization_specification: nil, format: nil, location: [], media_type: [], mesh: [], ordering_information: nil, people_represented: [], resource_query: [], issue: nil, searchable_text: nil, table_of_contents: nil, volume: nil, label: nil, relative_path: nil, import_url: nil, resource_type: [], creator: [], contributor: [], description: [], keyword: [], license: [], rights_statement: [], publisher: [], date_created: [], subject: [], language: [], identifier: ["single"], based_near: [], related_url: [], bibliographic_citation: [], source: [], access_control_id: "1f472c1b-6603-46ec-8606-8e9a4e3a542b", representative_id: nil, thumbnail_id: nil, rendering_ids: [], admin_set_id: "admin_set/default", embargo_id: nil, lease_id: nil>] irb(main):015:0> possible_duplicates.any? { |pd| pd.slug == slug } => false ```

next I tried to create works manually to see if the problem was resolved. it allowed me to create a work with an identifier and title of beat followed by a work with an identifier and title of beats. 🎉 however, while the works were created correctly, the beats title and identifier do not display correctly.

beat ``` bash irb(main):019:0> Text.where(slug: 'beat').count => 2 irb(main):022:0> Text.find '3b5cd0c5-f55a-45e9-837f-3574f9f86a90' => # ``` ![Image](https://user-images.githubusercontent.com/29032869/229914042-ae7b7a7e-0334-4a84-8d3c-3110d623beab.png)
beats ``` bash irb(main):020:0> Text.where(slug: 'beats').count => 2 irb(main):023:0> Text.find 'ab46bf3b-78cd-4db5-a08f-8d532149fac7' => # ``` ![Image](https://user-images.githubusercontent.com/29032869/229913400-66971fd6-06de-4d73-be4b-b79215d603f1.png)
alishaevn commented 1 year ago

when looking at the presenter for the "beats" work, several solr properties are saved as "beat" instead.

Image

alishaevn commented 1 year ago

looking for the solr document in the solr dashboard shows that the title and identifier values show "beats", which is correct. however, when I added an override for #initialize into Hyrax::WorkShowPresenterDecorator, and refreshed http://hyku.test/concern/texts/beats, the value of solr_document had the title and identifier values as "beat". 😩

it looks like the wrong solr doc is being passed to the presenter. maybe the same where method is being called somewhere and it's causing this issue?

def initialize(solr_document, current_ability, request = nil)
  binding.pry
  @solr_document = solr_document
  @current_ability = current_ability
  @request = request
end
solr dashboard ![Image](https://user-images.githubusercontent.com/29032869/230156950-375ebc67-448c-4881-a615-f7a6bd5aec22.png)
Hyrax::WorkShowPresenterDecorator#initialize ![Image](https://user-images.githubusercontent.com/29032869/230162160-bd53ac18-9036-40fc-9c60-18fed68d2a8f.png)
alishaevn commented 1 year ago

figured out the problem! ref: #182

alishaevn commented 1 year ago

reopening because I'm adding specs in a different pr.

alishaevn commented 1 year ago

@DiemBTran this is ready for staging qa! you can use "/app/samvera/hyrax-webapp/tmp/originals/two_up/two_up.csv" as the server path.

alishaevn commented 1 year ago

diem is out sick today so I've kicked off importer 2 on staging.

rachelihoward commented 1 year ago

Please tell me it’s not on our hyku-docker-dev site! I have an import still in process there – I started it on March 30 and it’s not yet complete. I was wondering what was taking so long.

From: Alisha Evans @.> Sent: Monday, April 10, 2023 10:54 AM To: scientist-softserv/louisville-hyku @.> Cc: Howard, Rachel @.>; Mention @.> Subject: Re: [scientist-softserv/louisville-hyku] works with a filename that includes the letter “s” before the filetype are skipped (Issue #172)

CAUTION: This email originated from outside of our organization. Do not click links, open attachments, or respond unless you recognize the sender's email address and know the contents are safe.

diem is out sick today so I've kicked off importer 2https://louisville-hyku-staging.notch8.cloud/importers/11?locale=en on staging.

— Reply to this email directly, view it on GitHubhttps://github.com/scientist-softserv/louisville-hyku/issues/172#issuecomment-1501914155, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A5TKJLAYSYCFI4RFP2IN3E3XAQNJPANCNFSM6AAAAAAUWUK2IU. You are receiving this because you were mentioned.Message ID: @.**@.>>

alishaevn commented 1 year ago

@rachelihoward do not fear, the import is on https://louisville-hyku-staging.notch8.cloud. 🙂

rachelihoward commented 1 year ago

Phew! Thanks for clarifying…

From: Alisha Evans @.> Sent: Monday, April 10, 2023 10:58 AM To: scientist-softserv/louisville-hyku @.> Cc: Howard, Rachel @.>; Mention @.> Subject: Re: [scientist-softserv/louisville-hyku] works with a filename that includes the letter “s” before the filetype are skipped (Issue #172)

CAUTION: This email originated from outside of our organization. Do not click links, open attachments, or respond unless you recognize the sender's email address and know the contents are safe.

@rachelihowardhttps://github.com/rachelihoward do not fear, the import is on https://louisville-hyku-staging.notch8.cloudhttps://louisville-hyku-staging.notch8.cloud/. 🙂

— Reply to this email directly, view it on GitHubhttps://github.com/scientist-softserv/louisville-hyku/issues/172#issuecomment-1501918552, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A5TKJLBPVNCGXCABRP25HY3XAQNXPANCNFSM6AAAAAAUWUK2IU. You are receiving this because you were mentioned.Message ID: @.**@.>>

alishaevn commented 1 year ago

the importer above failed because the deploy to main with the necessary code changes failed on april 6.

alishaevn commented 1 year ago

two_up.zip

after updating the volumes on staging and redeploying, I was able to successfully rerun the importer using the csv and files in the zip above. 🎉

Image

alishaevn commented 1 year ago

@rkuehn-uofl @rachelihoward please take a look at https://github.com/rkuehn-uofl/hyku/pull/25 and qa!

rkuehn-uofl commented 1 year ago

@alishaevn we're testing the latest code. CC: @rachelihoward

rachelihoward commented 1 year ago

@alishaevn we successfully tested this on our dev server (which was pretty much empty)! The soon-to-be-production server, with 81K plus items on it, has become so buggy that we can't successfully test anything on it.