CSV importer Part I

[DanielePalombo]

https://github.com/nebulab/solidus/issues/31

Rel https://github.com/nebulab/solidus/issues/31 First part

Make a CSV importer upload products, orders, and customers.

We think that the importer will be used only for the first import. So we can skip the check to update the entities.

Describe alternatives you've considered We can create a Converter from CSV to JSON to reuse the existing importers.

Additional context existing importers

[DanielePalombo]

Some details after meeting:

https://github.com/orgs/nebulab/projects/3#card-30533925

• same CSV formats as Shopify
• using ActiveJob
• other extensions can add their own jobs
• admin UI to upload CSV files
• log importing events and errors with solidus Log

owners: @blocknotes @DanielePalombo @elia

[blocknotes]

After some analysis with @DanielePalombo:

• Extension name: Solidus Importer
• Input: a Shopify CSV file
• Creates a CSV file entity (STI with 3 types) (columns: filename, state, messages, etc.)
• Worker which manage the CSV file entity
  • SolidusImporter::CSVFileBase#scan: create import rows entities (3 types STI)
  • SolidusImporter::CSVFileBase#process:
  • pass every import row to a import row worker
• Import row worker:
  • SolidusImporter::CSVRowBase#process
  • SolidusImporter::CSVRowBase#post_process

# => correspond to a CSV file
class SolidusImporter::CSVFileBase
  # STI with: CSVFileCustomers, CSVFileOrders, CSVFileProducts

  def scan # in after_create
    # create CSVRow entities
  end

  def pre_process
    # example: parse each CSVRow
    #   extract all taxons
    #   create missing taxons, with a reference: CSVRow <-> Taxon
  end

  def process
    # process every row
  end

  def post_process
    # example: send a notification
  end

  def check_finished
    # check if all rows finished, if yes:
    #   - change state to complete
    #   - call post_process
  end
end

# => correspond to a CSV row
class SolidusImporter::CSVRowBase
  # STI with: CSVRowCustomer, CSVRowOrder, CSVRowProduct

  attr_accessor :procs

  # reference to parent

  def initialize
    @context = nil
    @procs = []
  end

  def process
    @procs.each do |fun|
      fun.call(@context)
    end
  end

  def post_process
    # ... log or n-to-n: references to generated entities ...
    # call parent.check_finished
  end
end

[blocknotes]

New Admin Import section with:

• Import new file:
  • select data type (customers, products, orders) (infer it from filename?)
  • upload CSV file (processing will start after create or we will call it manually?)
• Imports list:
  • list running imports (poll for updates?)
  • list completed imports paginated (with state), clicking on a row open import details
• Import details:
  • show import details (filename, state, dates, number of completed / failed rows, etc.)
  • list import rows with pagination, clicking on a row open import row details
  • search / filter features
• Import Row details:
  • show import row details (state, dates, etc.)
  • list imported entities (use Spree::Log entities?)

Other features:

• notify when finished check box and email field? (per Import)

[blocknotes]

Extension project ➡️https://github.com/nebulab/solidus_importer