feat: JS and templated URL management in one input

atopper commented 1 month ago

Many users create unique import JS for specific templates such as 'blogs', 'news', 'events', etc. They keep a list of those URLs in files and then copy/paste them into the URLs field and then change the Transformation file URL field to match those templates (i.e. blog.import.js, or news.import.js). That is awkward.

Expected Behaviour

Matching the list of URLs (templates) to the JS script could be done many ways. For example:

For "Transformation file URL", the user enters: http://localhost:3001/tools/importer/blog.import.js
A new "Load URLs" button is clicked.
The script reads http://localhost:3001/tools/importer/blog.import.txt which contains 1 URL per line.
- In Workbench view: load the first (or a random one if CTRL is held with the click)
- In Bulk view: load the entire list into the field
If successful, the URL field is populated with the valid URLs found in the txt file.
If not, it leave the URL field as is.

FUTURE

Could crawling help set up the TXT files?

Actual Behaviour

There is a lot of copy/pasting and manual typing and awkwardness when managing/matching URLs and scripts.

Reproduce Scenario (including but not limited to)

Steps to Reproduce

Platform and Version

Sample Code that illustrates the problem

Logs taken while reproducing problem

kailasnadh790 commented 3 weeks ago

The current process of manually managing and matching URLs with transformation scripts is indeed cumbersome, especially for users handling multiple templates like blogs, news, and events. Automating the process through a "Load URLs" button that reads from a text file would streamline workflows significantly, reducing copy-pasting and potential for errors. This enhancement would make the feature more efficient and user-friendly, especially for bulk imports. Additionally, exploring future options like automated crawling for URL generation could be a valuable improvement to this functionality.

Also, currently in Helix Importer, we keep losing the URLs that we import as they are not saved anywhere. This approach would help in addressing that issue.

+1 for implementing this feature!

atopper commented 2 weeks ago

It would look a little like this quickly created video (no audio - basic idea) below.

If you want to try it out, do this command with a fairly recent recent aem-cli npm installation - version 16.7.11 or more recent: aem import --ui-repo=https://github.com/atopper/helix-importer-ui#442-url-management If you'd like to see this in the main product, please let a comment in this issue. Thanks.

https://github.com/user-attachments/assets/af49f8d0-442a-4095-9977-5074311cfc8c

adobe / helix-importer-ui