Debugger not working with SQL Server 2017 and Python

gbritton1 commented 6 years ago

Environment data

VS Code version: 1.21.1
Extension version (available under the Extensions sidebar): 2018.2.1- OS and version: XXX
Python version (& distribution if applicable, e.g. Anaconda): XXX
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): XXX
Relevant/affected Python packages and their versions: XXX

Actual behavior

Debugger does not stop at breakpoints
Program crashes in Microsoft code
Program runs fine from the CLI

Expected behavior

Debugger stops at defined breakpoints
Program does not crash

Steps to reproduce:

On an machine with SQL Server 2017 and Microsoft Machine learning with Python installed, run the attached program

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

There is no output panel called Python

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help)


console.ts:136 [Extension Host] (node:5764) DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
t.log @ console.ts:136

traceback

Express Edition will continue to be enforced. Elapsed time: 00:00:01.6467579 Error: Exception: 'resourcePath' Error in processing machine learning request. TlcBridge error: Error: Exception: 'resourcePath' Error in processing machine learning request.TlcBridge function failed. Please see the console output for more information. Traceback (most recent call last): File "c:\Users\Jerry\Documents\Pluralsight\Getting Started with Python on SQL Server\05 MicrosoftML\Demo\probably_pizza_featurize.py", line 23, in get_sentiment(cols=dict(scores="comment")) File "C:\Program Files\Microsoft SQL Server\MSSQL14.SQL_PYTHON\PYTHON_SERVICES\lib\site-packages\microsoftml\modules\featurize.py", line 209, in rx_featurize telemetry_info=telemetry_info File "C:\Program Files\Microsoft SQL Server\MSSQL14.SQL_PYTHON\PYTHON_SERVICES\lib\site-packages\microsoftml\utils\entrypoints.py", line 150, in run ret = tlc_bridge(maml_code=maml_code, analysis_type="RunGraph", *kargs) File "C:\Program Files\Microsoft SQL Server\MSSQL14.SQL_PYTHON\PYTHON_SERVICES\lib\site-packages\revoscalepy\utils\RxFunctionUtils.py", line 8, in wrapper return func(args,**kwds) File "C:\Program Files\Microsoft SQL Server\MSSQL14.SQL_PYTHON\PYTHON_SERVICES\lib\site-packages\microsoftml\utils\tlc_bridge.py", line 171, in tlc_bridge functionname=function_name, params=call_parameters) File "C:\Program Files\Microsoft SQL Server\MSSQL14.SQL_PYTHON\PYTHON_SERVICES\lib\site-packages\revoscalepy\RxSerializable.py", line 358, in rx_native_call ret = px_call(functionname, params) RuntimeError: revoscalepy function failed. 2018-03-15_19-56-17 2018-03-15_19-56-05

probably_pizza_featurize.zip

DonJayamanne commented 6 years ago

Please could you try debugging a simple python script such as the following:
```
print("Hello World")
```
When debugging the python program, where does SQL Server 2017 come into the picture? Is this used just to pull information from?
Please could you upload the contents of your launch.json file

gbritton1 commented 6 years ago

When running print("hello world"), SQL Server comes into the picture since it is the version of Python shipping with SQL server that I am running:


PS > & 'C:\Program Files\Microsoft SQL Server\140\PYTHON_SERVER\python.exe'
Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul  5 2016, 11:41:13) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

In the program that causes the errors, it is using the Microsoft packages revoscalepy and microsoftml. with local data.

Uploaded launch.json

launch.zip

DonJayamanne commented 6 years ago

Does debugging a simple hello world work? (Do break points work)
Does running the program using the Run without debugging option work?

gbritton1 commented 6 years ago

Running a simple Hello World works fine Run without debugging fails the same way, for the sample program (though it works fine when run from powershell or cmd.exe)

Additionally, the imports take a long time (20-30 seconds) when run from VS Code, but not from other launchers (e.g. cmd.exe or pycharm)

brettcannon commented 6 years ago

Since none of us have SQL Server installed I have reached out to the SQL Server team to see if they could provide any insight.

gbritton1 commented 6 years ago

I should point out that this is, by definition, Windows only, since SQL Server Python support only runs on Windows.

Also, I have talked to the SQL Server team. They basically said, "Not my problem!" Hate to get into finger-pointing here.

santina commented 6 years ago

Hi @gbritton1, could you try using C:\Program Files\Microsoft SQL Server\MSSQL14.SQL_PYTHON\PYTHON_SERVICES\ as your Python path? Thanks! Let us know if that works out.

gbritton1 commented 6 years ago

That is what I'm using. That's how I got the error I reported. FWIW the program would have crashed much earlier, since it would not have been able to do any imports, without that being the path

santina commented 6 years ago

From your earlier post where you attached the launch.zip file, it looks like you were using "C:\Program Files\Microsoft SQL Server\140\PYTHON_SERVER\python.exe" instead of "C:\Program Files\Microsoft SQL Server\MSSQL14.SQL_PYTHON\PYTHON_SERVICES\python.exe". That's why I was thought maybe that's an issue. Could you confirm which one you're using? Thanks!

gbritton1 commented 6 years ago

I'm using the first one. I'll check the second one later. Not sure it exists

gbritton1 commented 6 years ago

So, that was interesting. Running from C:\Program Files\Microsoft SQL Server\140\PYTHON_SERVER works both from the command line and from VS Code (at least, at the moment). Running from C:\Program Files\Microsoft SQL Server\MSSQL14.SQL_PYTHON\PYTHON_SERVICES fails in both cases

I've been slavishly following the docs for Python in SQL Server and never noticed that it could be installed twice, but it stands to reason. One is for the instance (to be used for in-database processing), the other is for the standalone installation. Which is interesting. If you only install the standalone version, which is typical for a desktop (where you don't need an instance), you only get C:\Program Files\Microsoft SQL Server\140\PYTHON_SERVER which thankfully works.

I also compared the contents of the two directories. They are identical. So, that nails down the repro (though, looking at my OP (the traceback) I see it did show the instance directory, not the standalone one.

santina commented 6 years ago

It looks like you cannot break at that line of code that creates the tuple if it's separated into multiple lines. I see this in both PyCharm and VSCode with your code. If you make it into a single line, it'd stop at the line Comments. This is likely just a debugger thing and does not have to do with microsoftml.

As for crashing. It sounds like with C:\Program Files\Microsoft SQL Server\140\PYTHON_SERVER, your script works in both cmd and VS Code. While using C:\Program Files\Microsoft SQL Server\MSSQL14.SQL_PYTHON\PYTHON_SERVICES, the code fails with errors you showed in the traceback.

It may be that microsoftml package in PYTHON_SERVER\ contains the pre-trained model needed for the sentiment analysis while PYTHON_SERVICES\ does not. The pre-trained model are in Lib\site-packages\microsoftml\mxlibs<modelname>_updated.model for Python and \mxlibs\x64<modelname>_updated.model for R. (See reference). Let me know if this is the case.

santina commented 6 years ago

Sorry, the model you need would be microsoftml/mxLibx/pretrained.model. You can obtained the pretrained models here by clicking on MLM_9.3.0.0_1033.cab

gbritton1 commented 6 years ago

Brilliant! The models are indeed installed in the standalone tree but not in the instance tree. Copying those across made it work properly. You know, if you hadn't jumped in, I would not have noticed that there were two directories at play here. Thanks a ton!.

microsoft / vscode-python