lblod / app-lblod-harvester

Harvesting Self Service
MIT License
1 stars 4 forks source link

[Harvesting multi-pages with auth] Adding authentication-configuration has-one relationship #30

Closed Poltergeistz closed 1 year ago

Poltergeistz commented 1 year ago

DL-4860

This PR is responsible for ensuring the harvest flow for several pages with authentication for normal jobs (not scheduled ones).

It works for no auth, basic authentication for single and multiple pages. Note : still need to test with oauth2 .

This is linked to other PRs :

How to test :

  1. Frontend : Pull and checkout feature/add-authentication-configuration-to-harvesting-collection
  2. Backend [app-lblod-harvester] : Pull and checkout feature/add-auth-conf-to-collection
  3. Pull and checkout feature/attach-and-remove-credentials from harvest-collector service run docker build -t your_image_name:tag .
  4. Backend [app-lblod-harvester] : add a docker-compose.override.yml file with your built image of harvest-collector-service
    
    version: '3.7'

services: harvest-collector: image: your_image_name


5. Backend [app-lblod-harvester] : docker-compose -f docker-compose.yml -f docker-compose.dev.yml -f docker-compose.override.yml up -d
6. Backend [app-lblod-harvester] : log 
-  `docker-compose logs -f --tail=all harvest-collector`
-  `docker-compose logs -f --tail=all harvesting-download-url`
-  `docker-compose logs -f --tail=all  harvesting-import`
7. Frontend : start with `ember s --proxy http://localhost:80`
8. Go to create a new job
9. Pick Harvest URL
10. Provide the url endpoint
11. Choose your security scheme
12. Input login info
13. Note : For non auth you don't have to provide authentication-configuration 
14. Schedule the job