missing web dashboards

[mhow2]

It seems we miss some dashboards with the current scava deployment. We only get scavaProject in the list from kibiter UI / dashboard In addition, we're facing the follow error (which could be related):

Traceback (most recent call last):
  File "./scava2es.py", line 303, in <module>
    elastic.bulk_upload(scava_metrics, "uuid")
  File "/usr/local/lib/python3.7/site-packages/grimoire_elk/elastic.py", line 185, in bulk_upload
    for item in items:
  File "./scava2es.py", line 271, in fetch_scava
    for project_scava in scava.fetch():
  File "/usr/local/lib/python3.7/site-packages/perceval/backend.py", line 128, in fetch
    yield self.metadata(item)
  File "/usr/local/lib/python3.7/site-packages/perceval/backend.py", line 163, in metadata
    'updated_on': self.metadata_updated_on(item),
  File "/usr/local/lib/python3.7/site-packages/perceval/backends/scava/scava.py", line 179, in metadata_updated_on
    updated = item['executionInformation']['lastExecuted']
KeyError: 'lastExecuted'

[valeriocos]

The error you report isn't related to the missing dashboards (I'll submit a PR to upload the other dashboards soon), but to an item (see below) extracted from the SCAVA API without execution time information. In particular, the attribute lastExecuted in executionInformation is missing, such a field (commonly present in other items) is needed by Perceval to process the data.

{'_type': 'org.eclipse.scava.repository.model.github.GitHubRepository', 
'parent': {}, 
'executionInformation': {
   '_id': '000ad112-9808-47b1-accf-5119f6c6dd81', 
  '_type': 'org.eclipse.scava.repository.model.ProjectExecutionInformation', 
  'storage': {
        '_id': 'c42f2088-14ff-4766-8ed5-a92ce4a5287f', 
        '_type': 'org.eclipse.scava.repository.model.LocalStorage', 
       'path': '/home/aboudeffa/scava/netty'}, 
'metricProviderData': [], 
'inErrorState': False}, 
'vcsRepositories': [
     {'_id': '6cfd78d1-7398-4581-b630-0c13d57b561c', 
      '_type': 'org.eclipse.scava.repository.model.vcs.git.GitRepository', 
      'persons': [], 
      '_superTypes': ['org.eclipse.scava.repository.model.vcs.git.VcsRepository'], 
      'url': 'https://github.com/netty/netty.git', 
      'username': 'xxx', 'password': 'xxx'}
], 
'communicationChannels': [], 
'bugTrackingSystems': [
       {'_id': '32c24274-1c9b-4b84-9f53-eb183d43d9cb', 
        '_type': 'org.eclipse.scava.repository.model.github.GitHubBugTracker', 
       'persons': [], 
       'issues': [], 
       '_superTypes': ['org.eclipse.scava.repository.model.github.BugTrackingSystem'], 
      'url': 'https://api.github.com/repos/netty/netty/issues', 
      '_ossmeterId': 'netty/netty:github', 
      'user': 'netty', 
      'repository': 'netty', 
      'login': 'xxx', 
      'password': 'xxx'}
], 
'companies': [], 
'languages': [], 
'contents': [], 
'downloads': [],
 'forks': [],
 'milestones': [], 
'description': 'Netty project - an event-driven asynchronous network application framework', 
'full_name': 'netty/netty', 
'name': 'netty', 
'_private': False, 
'fork': False, 
'git_url': 'git://github.com/netty/netty.git', 
'html_url': 'https://github.com/netty/netty', 
'clone_url': 'https://github.com/netty/netty.git', 
'homePage': 'http://netty.io/', 
'token': '753d13405face6c01f919ce7f14c187a1adf511d', 
'ssh_url': 'git@github.com:netty/netty.git', 
'svn_url': 'https://github.com/netty/netty', 
'size': 52630, 
'shortName': 'netty'
}

[tdegueul]

Can confirm I have the same issue locally. Could Perceval be updated to continue gracefully when lastExecuted is missing?

[valeriocos]

Hi @tdegueul I have submitted a PR to SCAVA: https://github.com/crossminer/scava/pull/66. It includes the fix for the Perceval backend and other changes to scava2es. If you want, I can split the PR in two.