microsoft / sqlmlutils

Utility functions for easier usage of SQL Server Machine Learning Services
Other
32 stars 33 forks source link

Add external language support #82

Closed JonathanZhu11 closed 4 years ago

JonathanZhu11 commented 4 years ago

With SQL Server 2019 CU3+ there are "language extensions" which are custom runtimes of R or Python that the user can create. This PR makes sqlmlutils work with sp_execute_external_script calls to R/Python that are called something else because they are an external language.

We need to disable a few tests because they are too slow to run in the CI. I will need to rework them to be faster in a later PR.

https://github.com/microsoft/sql-server-language-extensions

There are also some minor changes for addressing packaging bugs: we have a max wheel version (which still has pep425tags) and we use sysconfig for get_platform.

Aniruddh25 commented 4 years ago

suggestion, if not already done, please create a task to reenable the disabled tests #Closed

Aniruddh25 commented 4 years ago

I notice the sqlmlutils package version on Python is different than in R. Is that intentional ? #Closed

Aniruddh25 commented 4 years ago
    assert "exists on server. Set upgrade to True" in output.getvalue()

make sure to check version even in this new test and verify we get old_version ? basically, the following check_version() function from the test below can be made a common function with packagename as parameter and both tests can call this.

    def check_version(packagename):
        import packagename as cp
        return cp.__version__

    oldversion = pyexecutor.execute_function_in_sql(check_version) #Closed

Refers to: Python/tests/package_management_pypi_test.py:104 in 428704f. [](commit_id = 428704f5127f7643bf4dad6b4c07f1792cfffb87, deletion_comment = False)

Aniruddh25 commented 4 years ago

pyexecutor = SQLPythonExecutor(connection)

Looks to me these tests haven't been updated to use myPy yet, correct ?

You can add another task to add new tests using myPy/myR itself #Closed


Refers to: Python/tests/package_management_pypi_test.py:15 in 428704f. [](commit_id = 428704f5127f7643bf4dad6b4c07f1792cfffb87, deletion_comment = False)

JonathanZhu11 commented 4 years ago

It's... sort of intentional. I was planning to update R to version 1.0 when we got it into CRAN but CRAN is much harder to get into than PyPI (they have a lot more requirements). Also, matching the two versions doesn't necessarily make sense - if I make a bug fix in one language that shouldn't update the version in the other.

I should probably make them the same major version though.

Also, this change might be a major version change, so I may update both of them to v2.0.


In reply to: 705138493 [](ancestors = 705138493)

JonathanZhu11 commented 4 years ago

pyexecutor = SQLPythonExecutor(connection)

i have made a task


In reply to: 705259892 [](ancestors = 705259892)


Refers to: Python/tests/package_management_pypi_test.py:15 in 428704f. [](commit_id = 428704f5127f7643bf4dad6b4c07f1792cfffb87, deletion_comment = False)