Closed a20121248 closed 1 year ago
smlindauer
commented
1 year ago Hey @a20121248,
The truncated logs from the score test make it difficult to debug what is going on during the execution within pymas, but my suspicion would be that the DS2 wrapper for scoring in MAS is not being defined correctly. This would likely be a problem with register_model() function's generation of the DS2 code.
smlindauer
commented
1 year ago I replicated your error on a SAS Viya 3.5 server with SAS Model Manager, but was able to get score the model by making the following modifications in the SAS Model Manager UI:
dmcas_epscorecode.sas and dmcas_packagescorecode.sas filespython_wrapper.py file to have the role of "Score code"score.sas which has the "Score code" role and the model should be scorable with the iris.csv dataset found in the sasctl /examples/data folder.Alternatively, the pzmm submodule utilizes the "Generate DS2 score code from Python code" endpoint for SAS Viya 3.5 models so that sasctl does not have to generate the DS2 wrapper and should not see these same errors. We are currently working on refactoring the register_model() to utilize the pzmm functionality, but that is not likely to be available until the next release.
a20121248
commented
1 year ago Hi @smlindauer
Thank you for your help. I did all your steps and now I got a score.sas
I only have 2 questions.
Unfortunately, when I ran an scoring test, I got a different error.
If it helps, here I attached the files results:
testresult_code.txt
testresults_log.txt
a20121248
commented
1 year ago Regarding to pzmm, I have problems in the last step but not sure if it worth to open another Issue. I've got this error with Viya 3.5. I followed this tutorial HMEQ Dataset : Build and Import Trained Binary Classification Models into SAS Model Manager
smlindauer
commented
1 year ago Hi @smlindauer Thank you for your help. I did all your steps and now I got a
score.sas. I only have 2 questions.
- First, in your model properties (after click to "Generate DS2 Score code from Python code"), the Score code type should be DS2 package, right?
Correct. Since the generated score.sas code is set up for scoring in MAS (which is correlated to DS2 Package).
- Second, in the Project Variables (not model), the msg variable output is necessary, right?
I believe if you want to run the model through the Performance Monitoring, then that change will be necessary, but for score tests it is not needed.
Unfortunately, when I ran an scoring test, I got a different error.
The testResults_code.txt you are seeing is nothing like the code that I am seeing. It almost looks like the score code is treating the model like an Astore instead of running the wrapped DS2 code. I am not sure why that is occurring and could be an issue to take up with SAS Technical Support.
I have attached the model I created and modified from my replication of your error to see if that can provide some insights. Iris Regression.zip
smlindauer
commented
1 year ago Regarding to pzmm, I have problems in the last step but not sure if it worth to open another Issue. I've got this error with Viya 3.5. I followed this tutorial.
In regards to the error you are seeing from the pzmm notebook, it looks like there is a error occurring when attempting to upload the score code generated after importing the model to MM. I will need to see the error response from the server in order to confirm my suspicion of the error.
The fact that it seems to be failing at the call to /modelRepository/models/<UUID>/scoreResources, but is not being caught in the 406 try/except block leads me to guess that there may be a permission issue with access to the model resource directories. If a 403 or other error code is returned then that is likely the problem and you will need to contact your server administrator to adjust your permissions or the permissions of the model resource directory.
In order to enable debug logging, run the following in a cell above using your defined Session object:
sess.add_stderr_logger()
Then rerun the import_model() cell and isolate the last REST call that is logged with a failure code (>= 400).
a20121248
commented
1 year ago Regarding to the first method, I'll contact support to this specific issue, but now I understand how it works. I believe it's something with our Model Manager. By the way, yessss, with pzmm and the debug logging, as you said it's a 403 error in the last REST call 😃. I think is related to this Error 403 - registering SciKit-L model to MM
C:\ProgramData\Anaconda3\lib\site-packages\sasctl\pzmm\write_score_code.py:1060: UserWarning: Due to the ambiguity of the provided metrics and prediction return types, the score code assumes that a classification and the target event probability should be returned.
warn(
HTTP/1.1 POST https://plsnsasvawep01.datalake.local/modelRepository/models/ea7c8c0a-e067-463d-ab15-ce5fdeede7b5/contents?role=score
User-Agent: python-requests/2.24.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Content-Length: 2887
Content-Type: multipart/form-data; boundary=e6a534fd1c993f4df72e5f40e1534673
Authorization: Bearer [redacted]
Body:
(b'--e6a534fd1c993f4df72e5f40e1534673\r\nContent-Disposition: form-data; name'
b'="files"; filename="score_DecisionTreeClassifier.py"\r\nContent-Type: mult'
b'ipart/form-data\r\n\r\nimport math\nimport pickle\nimport pandas as pd\nimp'
b'ort numpy as np\nfrom pathlib import Path\n\nmodel_path = Path("/models/res'
b'ources/viya/ea7c8c0a-e067-463d-ab15-ce5fdeede7b5")\nwith open(model_path '
b'/ "DecisionTreeClassifier.pickle", "rb") as pickle_model:\n model = pi'
b'ckle.load(pickle_model)\n\ndef score(LOAN, MORTDUE, VALUE, YOJ, DEROG, DEL'
b'INQ, CLAGE, NINQ, CLNO, DEBTINC):\n "Output: EM_CLASSIFICATION, EM_EVE'
b'NTPROBABILITY"\n\n try:\n global model\n except NameError:\n'
b' model_path = Path("/models/resources/viya/ea7c8c0a-e067-463d-ab15-ce'
b'5fdeede7b5")\n with open(model_path / "DecisionTreeClassifier.pick'
b'le", "rb") as pickle_model:\n model = pickle.load(pickle_model'
b') try:\n if math.isnan(LOAN):\n LOAN = 18607.96979865'
b'772\n except TypeError:\n LOAN = 18607.96979865772\n try:\n '
b' if math.isnan(MORTDUE):\n MORTDUE = 73760.81719955898\n '
b'except TypeError:\n MORTDUE = 73760.81719955898\n try:\n i'
b'f math.isnan(VALUE):\n VALUE = 101776.04874145007\n except T'
b'ypeError:\n VALUE = 101776.04874145007\n try:\n if math.is'
b'nan(YOJ):\n YOJ = 8.922268135904503\n except TypeError:\n '
b' YOJ = 8.922268135904503\n try:\n if math.isnan(DEROG):\n '
b' DEROG = 0.2545696877380046\n except TypeError:\n DEROG = '
b'0.2545696877380046\n try:\n if math.isnan(DELINQ):\n D'
b'ELINQ = 0.4494423791821561\n except TypeError:\n DELINQ = 0.4494'
b'423791821561\n try:\n if math.isnan(CLAGE):\n CLAGE = '
b'179.7662751900467\n except TypeError:\n CLAGE = 179.766275190046'
b'7\n try:\n if math.isnan(NINQ):\n NINQ = 1.18605504587'
b'15597\n except TypeError:\n NINQ = 1.1860550458715597\n try:\n '
b' if math.isnan(CLNO):\n CLNO = 21.29609620076682\n exc'
b'ept TypeError:\n CLNO = 21.29609620076682\n try:\n if math'
b'.isnan(DEBTINC):\n DEBTINC = 33.77991534923519\n except Type'
b'Error:\n DEBTINC = 33.77991534923519\n\n input_array = pd.DataFra'
b'me([[LOAN, MORTDUE, VALUE, YOJ, DEROG, DELINQ, CLAGE, NINQ, CLNO, DEBTINC]],'
b'\n columns=["LOAN", "MORTDUE", "VALUE", "YOJ'
b'", "DEROG", "DELINQ", "CLAGE", "NINQ", "CLNO", "DEBTINC"],\n '
b' dtype=float)\n prediction = model.predict_proba(input'
b'_array)\n\n # Check for numpy values and convert to a CAS readable repr'
b'esentation\n if isinstance(prediction, np.ndarray):\n prediction'
b' = prediction.tolist()[0]\n\n if prediction[0] > prediction[1]:\n '
b' EM_CLASSIFICATION = "1"\n else:\n EM_CLASSIFICATION = "0"\n\n '
b'return EM_CLASSIFICATION, prediction[0]\r\n--e6a534fd1c993f4df72e5f40e1534'
b'673--\r\n')
HTTP 403 https://plsnsasvawep01.datalake.local/modelRepository/models/ea7c8c0a-e067-463d-ab15-ce5fdeede7b5/contents?role=score
Date: Fri, 14 Apr 2023 15:39:30 GMT
Server: Apache/2.4
Vary: Origin,User-Agent,Access-Control-Request-Method,Access-Control-Request-Headers
SAS-Service-Response-Flag: true
Content-Security-Policy: default-src 'self'; object-src 'none'; frame-ancestors 'self'; form-action 'self';
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
Strict-Transport-Security: max-age=31536000
Content-Type: application/vnd.sas.error+json;version=2;charset=UTF-8
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Keep-Alive: timeout=5, max=92
Connection: Keep-Alive
Transfer-Encoding: chunked
Body:
{'details': ['traceId: b67b741d5d84ddd8',
'path: '
'/modelRepository/models/ea7c8c0a-e067-463d-ab15-ce5fdeede7b5/contents'],
'errorCode': 403,
'httpStatusCode': 403,
'links': [],
'version': 2}And this is the Python error:
Model was successfully imported into SAS Model Manager as DecisionTreeClassifier with the following UUID: ea7c8c0a-e067-463d-ab15-ce5fdeede7b5.
---------------------------------------------------------------------------
HTTPError Traceback (most recent call last)
C:\ProgramData\Anaconda3\lib\site-packages\sasctl\core.py in request(verb, path, session, format, **kwargs)
2036
2037 if 400 <= response.status_code <= 599:
-> 2038 raise HTTPError(
2039 response.url, response.status_code, response.text, response.headers, None
2040 )
HTTPError: HTTP Error 403: {"errorCode":403,"details":["traceId: b67b741d5d84ddd8","path: /modelRepository/models/ea7c8c0a-e067-463d-ab15-ce5fdeede7b5/contents"],"links":[],"version":2,"httpStatusCode":403}Do you know what configuration I should ask or explain to the administrators? I'm more a user without admin rights doing things proactively.
Thank you again.
smlindauer
commented
1 year ago Regarding to the first method, I'll contact support to this specific issue, but now I understand how it works. I believe it's something with our Model Manager. By the way, yessss, with pzmm and the debug logging, as you said it's a 403 error in the last REST call 😃. I think is related to this Error 403 - registering SciKit-L model to MM
I agree, that his seems to be similar (if not the same as) to #39. The configuration in question has to do with the fact that the model repository location for model resources is set to administrator access.
Do you know what configuration I should ask or explain to the administrators? I'm more a user without admin rights doing things proactively.
I believe the following documentation which details the process of setting up the model resource directory for ASTORE models would be the corollary for properly setting up the Python model resources directory. Specifically, I would direct them to the Setting Permissions section of the article, which notes the adjustments needed for non-admin users to access the directory.
Hope that helps!
Hi experts:
Describe the issue Not able to score models
To Reproduce
Expected behavior The task should score a table.
Stack Trace If you're experiencing an exception, include the full stack trace and error message.
Version What version of sasctl are you using? 1.9.0
I have SAS Viya 3.5 and Python 3.6 in that server. If it's worth to know, I am able to use that Python through
proc fcmpbecause we set environment variables at some point.From another server with more compute resources to train models, I also have a Python container and I installed sasctl-1.9.0. I was able to connect to Viya (also SWAT is working) succesfully. I followed this Tutorial. However, I've got this problems:
First of all, in the example, I run all the code but the module.predict(x) is not working because I got None as a response. There's no more output or log available from Python.
Second, I noticed that my model was registered in SAS Model Manager, so I tried to upload the dataset and decided to do a manual Score run test. I see the log and I got this:
I hope you can help me. I am open to all ideas or guidance.