spcl / serverless-benchmarks

SeBS: serverless benchmarking suite for automatic performance analysis of FaaS platforms.
https://mcopik.github.io/projects/sebs/
BSD 3-Clause "New" or "Revised" License
150 stars 68 forks source link

Regression tests are not passing for Azure #178

Closed nurSaadat closed 1 year ago

nurSaadat commented 1 year ago

Describe the bug Cannot run Azure regression test.

To Reproduce Steps to reproduce the behavior:

  1. Go to project root folder
  2. Run the command ./sebs.py benchmark regression test --config config/example.json --deployment azure
  3. Scroll down to see the logs
  4. See the error

Expected behavior Regression tests are passed with no errors

Logs

[12:10:46.094593] HTTPTrigger-991d Invocation on URL https://110-dynamic-html-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:10:46.095187] HTTPTrigger-991d Output: 
110.dynamic-html fail on trigger: TriggerType.HTTP
[12:10:50.876687] HTTPTrigger-88d5 Invocation on URL https://120-uploader-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:10:50.877448] HTTPTrigger-88d5 Output: 
120.uploader fail on trigger: TriggerType.HTTP
[12:10:56.849252] HTTPTrigger-9d1e Invocation on URL https://210-thumbnailer-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:10:56.849833] HTTPTrigger-9d1e Output: 
210.thumbnailer fail on trigger: TriggerType.HTTP
[12:11:02.681845] HTTPTrigger-837c Invocation on URL https://220-video-processing-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:11:02.682334] HTTPTrigger-837c Output: 
220.video-processing fail on trigger: TriggerType.HTTP
[12:11:07.410633] HTTPTrigger-21ec Invocation on URL https://311-compression-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:11:07.411115] HTTPTrigger-21ec Output: 
311.compression fail on trigger: TriggerType.HTTP
[12:11:14.042761] HTTPTrigger-6ce0 Invocation on URL https://411-image-recognition-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:11:14.043221] HTTPTrigger-6ce0 Output: 
411.image-recognition fail on trigger: TriggerType.HTTP
[12:11:18.724323] HTTPTrigger-403d Invocation on URL https://501-graph-pagerank-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:11:18.724772] HTTPTrigger-403d Output: 
501.graph-pagerank fail on trigger: TriggerType.HTTP
[12:11:23.409691] HTTPTrigger-dd77 Invocation on URL https://502-graph-mst-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:11:23.410368] HTTPTrigger-dd77 Output: 
502.graph-mst fail on trigger: TriggerType.HTTP
[12:11:30.134037] HTTPTrigger-a5d8 Invocation on URL https://503-graph-bfs-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:11:30.134435] HTTPTrigger-a5d8 Output: 
503.graph-bfs fail on trigger: TriggerType.HTTP
[12:11:35.309965] HTTPTrigger-56c1 Invocation on URL https://504-dna-visualisation-python-3-7-70c51d6e.azurewebsites.net/api/handler failed!
[12:11:35.310453] HTTPTrigger-56c1 Output: 
504.dna-visualisation fail on trigger: TriggerType.HTTP

System (please complete the following information):

nurSaadat commented 1 year ago

@mcopik I have some updates on this issue

Figured out that adding the function key allows the function to be triggered successfully.

Now I am manually copying the function key from the Function >> Monitor >> Code + Test on Azure platform and substituting the function URL in serverless-benchmarks/regression-cache/[benchmark-name]/config.json

Is there any way to make parsing function code and putting it into config.json automatic?

mcopik commented 1 year ago

@nurSaadat Yes, the default behavior of Azure Functions changed, and HTTP triggers started requiring authorization.

I updated the master branch, and now SeBS should make function invocation without authorization. This should require no manual work on your part. Please pull the most recent version of the master branch and redeploy your functions, as the existing one will still require the key.

nurSaadat commented 1 year ago

Thank you so much, it works perfectly now. I think we can close this issue