DIRACGrid / diracx

The neXt DIRAC generation
GNU General Public License v3.0
8 stars 19 forks source link

Improvement to CS conversion #191

Open chrisburr opened 9 months ago

chrisburr commented 9 months ago
chrisburr commented 8 months ago

Can we call the cs sync when committing in legacy DIRAC so errors get reported?

chaen commented 7 months ago

note for me for later to sync the users:

curl -q -L -s  -u ${CLIENT_ID}:${CLIENT_SECRET}  -d grant_type=client_credentials -d scope=iam:admin.read     ${IAM_TOKEN_ENDPOINT} > /tmp/token.json

export AT=$(cat /tmp/token.json  | jq ".access_token")

import requests, json
import os
token = os.environ.get('AT')

headers = {"Authorization": f"Bearer {token}"}
iam_list_url = 'https://lhcb-auth.web.cern.ch/iam/account/search'
results = []
startIndex = 1
totalResults = 1000  # total number of users
itemsPerPage = 10
while(startIndex < totalResults):
    resp = requests.get(iam_list_url, headers=headers, params = {'startIndex':startIndex})
    resp.raise_for_status()
    data = resp.json()
    # These 2 should never change, but just to be sure...
    totalResults = data['totalResults']
    itemsPerPage = data['itemsPerPage']

    startIndex += itemsPerPage
    results.extend(data['Resources'])
with open('accounts.json', 'w+') as fp:
    fp.write(json.dumps(results))
chaen commented 7 months ago

Can we call the cs sync when committing in legacy DIRAC so errors get reported?

as discussed offline, the cs-sync can be ran at every commit, but the sync between IAM and the CS (for the users subs) takes too long ,and will be done in the VOMS2CSAgent