Setup/cleanup mdTranslator mini-app

jbrown-xentity commented 2 weeks ago

User Story

In order to get transformer to the latest code, harvest2.0 devs want the transform system setup with latest changes.

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]

Security Considerations (required)

None

Sketch

We want the following:

Get gsa fork of MDTranslator caught up with the latest release (see user story), and make a protected branch.
Setup mdTranslator datagov version into datagov-harvest-transformer](https://github.com/GSA/datagov-harvest-transformer)
Integrate this (via Dockerfile, something else?) into datagov-harvesting-logic
Make sure application is deployed into cloud.gov with latest version (is CI/CD setup?)

rshewitt commented 2 weeks ago

master is synced with upstream master. datagov is synced with upstream develop since most of the activity is on that branch and what i would assume we'd merge into when the time comes

rshewitt commented 2 weeks ago

we have deploy, publish (ghcr), and test ci/cd actions

rshewitt commented 2 weeks ago

removed mdtranslator in the compose file on this commit. can add back.

rshewitt commented 2 weeks ago

for anyone unfamiliar with package management in ruby ( not version management ) here's some info...

reidhewitt@Reids-MacBook-Pro mdTranslator % gem env 
RubyGems Environment:
  ...
  - INSTALLATION DIRECTORY: /home/bobsmith/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0
  - GEM PATHS:
     - /home/bobsmith/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0
     - /home/bobsmith/.gem/ruby/3.2.0
  ...

INSTALLATION DIRECTORY is the default directory where your gems are gonna be installed. GEM_PATHS are where the gems are searched from. in my case, I included a .ruby-version file in my root indicating to use ruby v3.2.2 which I had previously installed for another app. I wanted to use the same ruby version but since i'm using a different app i don't want them both to use the same installation directory.
if you want to install your gems in a folder in your root directory similar to node_modules you can configure bundle to do so.
- if you don't have a bundle config file create one in your root such that .bundle/config is a valid file path
then add BUNDLE_PATH: "your_folder_name" to the file. apparently, naming the folder "vendor" is common practice so BUNDLE_PATH: "vendor". if BUNDLE_PATH is configured it will be used to search for gems regardless of GEM_HOME or GEM_PATH (source)
you'll notice .bundle and vendor/ are included in the gitignore file (not a highlighted link because they're apart so find them)
install your gems via bundle install (you may or may not have to create "vendor" prior to installing. i already created it)
assuming everything is good your vendor folder should look something like...

rshewitt commented 1 week ago

info on ruby version management

rshewitt commented 1 week ago

IRB comes with your ruby installation and can be used to debug

insert a "breakpoint" by adding binding.irb on the line of your choice. here's an example of what that might look like...


From: /home/bobsmith/.rbenv/versions/3.2.2/lib/ruby/vendor_ruby/gems/3.2.0/gems/adiwg-mdtranslator-2.20.0.pre.beta.5/lib/adiwg/mdtranslator/writers/dcat_us/sections/dcat_us_contact_point.rb @ line 10 :

 5:       module Writers
 6:          module Dcat_us
 7:             module ContactPoint
 8: 
 9:                def self.build(intObj)
=> 10:                   binding.irb
11:                   resourceInfo = intObj[:metadata][:resourceInfo]
12:                   pointOfContact = resourceInfo[:pointOfContacts][0]
13:                   contactId = pointOfContact[:parties][0][:contactId]
14: 
15:                   contact = Dcat_us.get_contact_by_id(contactId)

irb(ADIWG::Mdtranslator::Writers:...):001> # execute your expression here...

GSA / data.gov