search

Azure / Azure-Functions

1.12k stars 199 forks source link

Port runtime to .NET Core #98

Closed fabiocav closed 7 years ago

fabiocav commented 7 years ago

This issue will track the .NET Core port activities for porting the runtime to .NET core. Note that netstandard 1.3 assemblies can be used on Azure Functions and this is fully supported.

We'll continue to update this list and details as we move along.

Have opened the following issues to track migration: WebJobs SDK - https://github.com/Azure/azure-webjobs-sdk/issues/1177 WebJobs SDK Extensions - https://github.com/Azure/azure-webjobs-sdk-extensions/issues/233 WebJobs Script (Functions Runtime) - https://github.com/Azure/azure-webjobs-sdk-script/issues/1576

Activities

Issues

imsam67 commented 7 years ago

Do we have an availability date for WebJobs SDK in .NET Core?

agnauck commented 7 years ago

Is there a public repo where we can contribute and monitor the netcore port?

fabiocav commented 7 years ago

Have opened the following issues to track migration: WebJobs SDK - https://github.com/Azure/azure-webjobs-sdk/issues/1177 WebJobs SDK Extensions - https://github.com/Azure/azure-webjobs-sdk-extensions/issues/233 WebJobs Script (Functions Runtime) - https://github.com/Azure/azure-webjobs-sdk-script/issues/1576

PRs to follow (starting with WebJobs SDK)

VenkateshSrini commented 7 years ago

Hi, Any update on where we are on this. I would like to run Azure web on Azure Linux box and host my work load here. Also I will be happy if we can containerize the same and also deploy them on cloud foundry as now Microsoft is a Gold Member in Cloud Foundry. Also now since .NET core 2.0 with .NET standard 2.0 supports most of the API, the transition is going to be lot more easier. Regards

lindydonna commented 7 years ago

@VenkateshSrini This work is in progress. You can see the detailed status by watching the issues that @fabiocav listed above: https://github.com/Azure/Azure-Functions/issues/98#issuecomment-306361546

fabiocav commented 7 years ago

The WebJobs SDK port has been completed and merged.

The basic extensions work has been done to enable work on the Script Runtime (Functions Runtime) to begin and that is currently in progress, tracked by Azure/azure-webjobs-sdk-script#1576

atrauzzi commented 7 years ago

Will this open up improved local development options at all?

lindydonna commented 7 years ago

@atrauzzi Yes. That was the main motivation for doing this work. We'll port the Azure Functions Core Tools to .NET Core and test on Mac/Linux.

VenkateshSrini commented 7 years ago

Hi, So please help me understand. If I were to take the current tool and make it run on Pure .NET core. What all should I do

Thanks and regards Venkatesh

On Thu, Jul 20, 2017 at 12:04 AM, Donna Malayeri notifications@github.com wrote:

@atrauzzi https://github.com/atrauzzi Yes. That was the main motivation for doing this work. We'll port the Azure Functions Core Tools to .NET Core and test on Mac/Linux.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Azure/Azure-Functions/issues/98#issuecomment-316477164, or mute the thread https://github.com/notifications/unsubscribe-auth/ABHPMNrKiDRPRTVbi5ShGBbgAqeYaqAgks5sPkwggaJpZM4LJZP5 .

fabiocav commented 7 years ago

@VenkateshSrini that is the work that is currently in progress.

atrauzzi commented 7 years ago

@lindydonna - Oh boy! You mean I won't need an active Azure subscription to tinker with functions?!? :)

+1000

galvesribeiro commented 7 years ago

Hello @lindydonna!

Any ETA on at least a very alpha-early-preview release? I got a new big project here that I'm trying to convince the customer to use Azure Functions over AWS Lambda but his developer base want to run .net core and they all use Macbook Pros with VSCode.

Please tell me something that I can keep them motivated to stick with Azure. How can we help to speed up things?

Thank you!

lindydonna commented 7 years ago

@galvesribeiro: @fabiocav can tell you the latest status, but I expect we'll have something you can try out within the next month.

atrauzzi commented 7 years ago

@galvesribeiro - Don't forget to make some noise about having access to a storage emulator on linux and OSX as well: https://feedback.azure.com/forums/217298-storage/suggestions/7048446-provide-storage-emulation-for-mac-and-linux

Right now the storage emulator only works on Windows. Which -- while not mandatory, Functions and storage definitely go together.

paulbatum commented 7 years ago

FYI there are a number of issues when using functions with the storage emulator and we don't recommend this configuration. You're better off developing locally using a real storage account.

Get Outlook for iOShttps://aka.ms/o0ukef

From: Alexander Trauzzi notifications@github.com Sent: Tuesday, August 8, 2017 7:49:51 PM To: Azure/Azure-Functions Cc: Paul Batum; Assign Subject: Re: [Azure/Azure-Functions] Port runtime to .NET Core (#98)

@galvesribeirohttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fgalvesribeiro&data=02%7C01%7Cpbatum%40microsoft.com%7C79be0f9b891d42dd0e2508d4ded14f3e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636378437957919349&sdata=8Mf041o9g52H0RE%2BgjGDdhrcVfppVqYYAiGZZhOy%2BR4%3D&reserved=0 - Don't forget to make some noise about having access to a storage emulator on linux and OSX as well: https://feedback.azure.com/forums/217298-storage/suggestions/7048446-provide-storage-emulation-for-mac-and-linuxhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ffeedback.azure.com%2Fforums%2F217298-storage%2Fsuggestions%2F7048446-provide-storage-emulation-for-mac-and-linux&data=02%7C01%7Cpbatum%40microsoft.com%7C79be0f9b891d42dd0e2508d4ded14f3e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636378437957919349&sdata=n2j3raZJdSLoU8JxCpAtvqD76eIti%2BTBzH54BLCY7Fo%3D&reserved=0

Right now the storage emulator only works on Windows. Which -- while not mandatory, Functions and storage definitely go together.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAzure%2FAzure-Functions%2Fissues%2F98%23issuecomment-321137716&data=02%7C01%7Cpbatum%40microsoft.com%7C79be0f9b891d42dd0e2508d4ded14f3e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636378437957919349&sdata=EjJgUg0KLZ%2F71OnaPXfOKaxnGJKVmkJsHG%2BWg3zgthE%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAApp2CC76n8Wk7HpbUP-nWqP2_MxWcuks5sWR5PgaJpZM4LJZP5&data=02%7C01%7Cpbatum%40microsoft.com%7C79be0f9b891d42dd0e2508d4ded14f3e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636378437957919349&sdata=IOaOi5f39DsZyYvE81C60pMHX3jjCooPMp6Z7tRwwK8%3D&reserved=0.

galvesribeiro commented 7 years ago

@atrauzzi Yes, I have an open (internal) issue to get storage emulator in a Docker Container. The current version "just don't work". The same as in DocumentDB emulator.

To avoid that problem, we implemented our own storage emulator following the required parts of the API for our use case and it works ok. We are thinking on when/how to publish it OSS so people can contribute it.

The current implementation of those emulators are terrible IMHO. The storage one use SQL Express in a "bad way". The DocumentDB use some of the Service Fabric components, which is not good as well.

Anyway, we were looking at how the current functions emulator works. It should not take too long to implement an emulator yourself if the "external" dependencies got mocked with a "local" implementation.

atrauzzi commented 7 years ago

@galvesribeiro - Totally agree. The current emulators make a lot of bad assumptions.

@paulbatum - That's not really viable for local development. It also imposes a cost and ends up sounding like a way to bill people for learning the platform. You need free, cross platform local options, period.

paulbatum commented 7 years ago

@atrauzzi I'm sorry to contradict you so directly but its really quite viable - using real storage accounts for local development is what the vast majority of our users do today (as per our recommendation). Now I'm not going to claim its ideal - I agree with you that a completely free local option would be better, but this is the state of things right now. Pointing functions users to the local emulator will just lead to a bad experience when they try to use features that rely on storage functionality that is not provided by the emulator. We actually updated our documentation recently to spell this out (see the note at the bottom of this section).

We'll revisit this recommendation if the storage emulator reaches full parity (or enough parity such that there are no known issues when using it with functions), and of course I'm not debating your point that the emulators should go cross platform - that would be just goodness.

galvesribeiro commented 7 years ago

Yeah... I agree with @paulbatum partially. Due to the very bad state of all Azure SDK emulators available, we ended up using real storage accounts. Ok, for a small 25 people team. For large corporations, it is complicated. All AWS emulators are xplat (sadly in Java or Python but they are) and full feature parity. If MSFT want to bring more and more users to Azure, and now with .net core broadly available, they should rethink those emulators.

As I said, we are investigating the efforts to bring the emulators we crated and add Functions one as well to OSS as an alternative to "official" ones.

xt0rted commented 7 years ago

The emulators are an important part of development just as sql server express is, and every time I see someone say they aren't supported for X go use Azure it really annoys me because that's not a viable option for everyone or a good experience. This type of response is exactly what prompted me to send a PR to the WebJobs SDK that let you use the emulator with it.

Using the emulators prevents me from having to rely on having an internet connection and from committing real connection strings. This also means I don't have to think about accidentally committing connection strings if they're defaulted to something like << upload storage endpoint >> which I then have to set to my own on checkout.

The lack of a Service Bus emulator has caused me to skip over using the product entirely and to just use multiple Storage Queues. It's not the best technical solution, but it's the easiest development experience which is going to win every time.

fabiocav commented 7 years ago

A quick update on this work:

/cc @paulbatum , @lindydonna , @ahmelsayed , @christopheranderson

galvesribeiro commented 7 years ago

@fabiocav I'm trying the core version of the CLI, but I'm not seeing options to create a class library from it so I can use pre-compiled functions. Any advices?

galvesribeiro commented 7 years ago

@lindydonna / @fabiocav I would appreciate any help. The core CLI indeed work on OSX if you are using the .csx. But that means, no debugger for C#... If you guys could let us know how to build and run a pre-compiled functions assembly and have minimal debugging functionality, I would appreciate that. Thank you.

galvesribeiro commented 7 years ago

Guys?

imsam67 commented 7 years ago

I agree that lack of communication isn't helpful.

lindydonna commented 7 years ago

@galvesribeiro We haven't officially announced the preview, so we don't have a lot of instructions. Expect a blog post in a couple of weeks with more info.

To build a class library, use dotnet build and dotnet publish. You need to manually author function.json.

galvesribeiro commented 7 years ago

@lindydonna

image

Build and publish a regular dll... Ok... What about publishing to Azure Function? Build locally is fine but we aren't able to publish it (look the picture). I built the function.json manually and no lucky.

galvesribeiro commented 7 years ago

@lindydonna

It would take no more than 10m for someone on your team which is actively working on it, to simple create a dummy hello-functions-world using the core CLI and a netstandard2.0 project or leave a quick note here on how we should create it.

It is frustrating to say at least being blocked... It is just making more people go to AWS Lambda because of this lack of (simple) communication...

lindydonna commented 7 years ago

@galvesribeiro To answer your question, simply create a netstandard2.0 class library with the dotnet CLI and do the regular "dotnet build" and "dotnet publish" steps. Then just launch the CLI from the publish directory.

Publish doesn't work, and the Functions runtime in Azure does not yet work with .NET Core 2.0 projects (but this will work soon).

So, we intentionally did not provide a lot of instructions because the experience is not ready for most to try. This is mainly for people who are familiar with the codebase and are ready to deal with the rough edges.

We'll have an announcement on our blog with detailed instructions in the next few weeks.

galvesribeiro commented 7 years ago

@lindydonna

To answer your question, simply create a netstandard2.0 class library with the dotnet CLI and do the regular "dotnet build" and "dotnet publish" steps. Then just launch the CLI from the publish directory.

That isn't compatible with your next statement:

Publish doesn't work, and the Functions runtime in Azure does not yet work with .NET Core 2.0 projects (but this will work soon).

Why would someone attempt to even use it if they can't deploy it?

Anyway, thank you for the reply. I guess we will need to move to AWS Lambda then.

goofballLogic commented 7 years ago

Dear G-d why would you not support .NET Core first? The same people who are naturally interested in Serverless technologies are the ones who will be using .NET Core.

galvesribeiro commented 7 years ago

@goofballLogic

I asked myself very same question...

SimonLuckenuik commented 7 years ago

Azure functions is based on Web Jobs SDK which was not originally based on .NET Core. Considering that there is no Linux based Azure Functions, support for .NET Core is mainly for non-Windows development environments.

davidebbo commented 7 years ago

@SimonLuckenuik right, that's exactly the reason it started as non-Core (plus Core was too immature when we started). But from 2.0 on, it'll be all Core, and work cross platform.

tjrobinson commented 7 years ago

@davidebbo Is there an ETA for 2.0 yet? I know you don't want to give dates but are we talking days, weeks, months? Thanks.

christopheranderson commented 7 years ago

Today! Announcement blogs:

nelak commented 7 years ago

@christopheranderson Does this also mean node worker has been upgraded to 8.5? Also I've tried npm i -g azure-functions-core-tools@core in wsl but it fails the following message:

npm ERR! code EBADPLATFORM
npm ERR! notsup Unsupported platform for azure-functions-core-tools@1.0.0-beta.100: wanted {"os":"win32","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! notsup Valid OS:    win32
npm ERR! notsup Valid Arch:  any
npm ERR! notsup Actual OS:   linux
npm ERR! notsup Actual Arch: x64

Any ideas on how to bypass the platform check?

ahmelsayed commented 7 years ago

@nelak what version of npm and node are you using? (npm --version, node --version)

it seems your npm is trying to install azure-functions-core-tools@1.0.0-beta.100, which is a pretty old version. It's not even the current 1.x version, the current 1.x version is 1.0.4.

what do you get if you do npm view azure-functions-core-tools? also you can try to explicitly install the latest @core version by doing npm i -g azure-functions-core-tools@2.0.1-beta1.15

nelak commented 7 years ago

@ahmelsayed Just checked and I was on node v8.0.0, not with v8.5.0, upgrading fixed the issue. I'm sending the requested info for v8.0.0 just in case it's useful: Trying to do npm i -g azure-functions-core-tools@2.0.1-beta1.15 would still resolve to azure-functions-core-tools@1.0.0-beta.100

npm --version 5.3.0

npm view azure-functions-core-tools

{ name: 'azure-functions-core-tools',
  description: 'Azure Functions Core Tools',
  'dist-tags': { latest: '1.0.4', core: '2.0.1-beta1.15' },
  versions:
   [ '1.0.0-beta.95',
     '1.0.0-beta.96',
     '1.0.0-beta.97',
     '1.0.0-beta.98',
     '1.0.0-beta.99',
     '1.0.0-beta.100',
     '1.0.0',
     '1.0.1',
     '1.0.2',
     '1.0.3',
     '1.0.4',
     '2.0.0',
     '2.0.1-beta.1',
     '2.0.1-beta.2',
     '2.0.1-beta.3',
     '2.0.1-beta.4',
     '2.0.1-beta.5',
     '2.0.1-beta.6',
     '2.0.1-beta.7',
     '2.0.1-beta.8',
     '2.0.1-beta.9',
     '2.0.1-beta.10',
     '2.0.1-beta.11',
     '2.0.1-beta.12',
     '2.0.1-beta.13',
     '2.0.1-beta.14',
     '2.0.1-beta1.15' ],
  maintainers:
   [ 'ahmelsayed <ahmelsayed@outlook.com>',
     'mattmason03 <mamaso@microsoft.com>',
     'windowsazure <azuresdk@outlook.com>' ],
  time:
   { modified: '2017-09-24T01:21:58.149Z',
     created: '2017-05-03T01:21:59.910Z',
     '1.0.0-beta.95': '2017-05-03T01:21:59.910Z',
     '1.0.0-beta.96': '2017-05-06T00:28:12.344Z',
     '1.0.0-beta.97': '2017-05-16T00:36:46.444Z',
     '1.0.0-beta.98': '2017-06-05T22:49:20.702Z',
     '1.0.0-beta.99': '2017-06-06T17:44:30.500Z',
     '1.0.0-beta.100': '2017-06-30T01:11:17.967Z',
     '1.0.0': '2017-08-11T20:16:18.849Z',
     '1.0.1': '2017-08-11T20:26:41.323Z',
     '2.0.0': '2017-08-24T00:42:42.553Z',
     '2.0.1-beta.1': '2017-08-24T00:52:25.874Z',
     '2.0.1-beta.2': '2017-08-24T01:39:42.781Z',
     '2.0.1-beta.3': '2017-08-24T03:59:28.009Z',
     '2.0.1-beta.4': '2017-08-24T19:53:37.771Z',
     '1.0.2': '2017-08-24T20:00:32.643Z',
     '2.0.1-beta.5': '2017-08-24T20:48:29.592Z',
     '2.0.1-beta.6': '2017-08-24T20:50:42.136Z',
     '2.0.1-beta.7': '2017-08-24T22:34:29.823Z',
     '2.0.1-beta.8': '2017-08-25T21:17:18.102Z',
     '2.0.1-beta.9': '2017-08-28T23:39:47.614Z',
     '2.0.1-beta.10': '2017-08-31T05:39:11.901Z',
     '2.0.1-beta.11': '2017-08-31T23:24:58.043Z',
     '1.0.3': '2017-09-08T20:24:56.964Z',
     '2.0.1-beta.12': '2017-09-12T00:06:54.866Z',
     '1.0.4': '2017-09-15T20:42:16.101Z',
     '2.0.1-beta.13': '2017-09-23T01:13:19.707Z',
     '2.0.1-beta.14': '2017-09-24T00:45:29.946Z',
     '2.0.1-beta1.15': '2017-09-24T01:21:58.149Z' },
  homepage: 'https://github.com/Azure/azure-functions-cli#readme',
  repository:
   { type: 'git',
     url: 'git+https://github.com/Azure/azure-functions-cli.git' },
  author: 'Microsoft',
  bugs: { url: 'https://github.com/Azure/azure-functions-cli/issues' },
  license: 'MIT',
  readmeFilename: '',
  users: { lastexile: true, 'ivan.marquez': true },
  version: '1.0.4',
  scripts: { postinstall: 'node lib/install.js' },
  preferGlobal: true,
  bin:
   { func: 'lib/main.js',
     azfun: 'lib/main.js',
     azurefunctions: 'lib/main.js' },
  os: [ 'win32' ],
  dependencies:
   { chalk: '^2.1.0',
     'command-exists': '^1.2.2',
     tmp: '^0.0.33',
     unzipper: '^0.8.9' },
  dist:
   { integrity: 'sha512-I2MpcThfchxX6o6k8ndDcvY2zTgYO6xs619R+7RWUbYnUsJRqh40VYUjmColsg6yVf5Erqb8kdsrLJ9DF2OyUw==',
     shasum: '0752a0b5b048f55fd955aa8b40874822b5a4d7cc',
     tarball: 'https://registry.npmjs.org/azure-functions-core-tools/-/azure-functions-core-tools-1.0.4.tgz' },
  directories: {} }
imsam67 commented 7 years ago

I installed the latest version of VS tools by running npm install -g azure-functions-core-tools. I also created a new profile under Properties > Debug and pointed it to the func.exe under the node_modules/azure-functions-core-tools. What else do I need to do to create a new Azure Function targeting .NET Core? When I created a new function in VS 2017, it still seems to be targeting .NET Framework 4.6.1.

paulbatum commented 7 years ago

@imsam67 these bits are in preview so you won't get them as a normal update. See this blog post for more details.

ahmelsayed commented 7 years ago

@nelak you're right it looks like @mamaso added node >=8.4.0 and I didn't notice that.

mamaso commented 7 years ago

Interesting - iirc npm removed the explicit errors around invalid engines / node versions. I added the engines field as documentation / a guideline, but if it's causing these strange resolution failures we should pull it out.

mattwoberts commented 7 years ago

@fabiocav Apologies if I'm posting in the wrong issue... Does this issue cover the work required to get an asp.net core 2 web app with webjobs deployed to azure? In ASP.NET, we'd use Microsoft.Web.WebJobs.Publish and deployment was fairly automated. In core - this doesn't seem to be the case, it looks like something we need to do manually. Could you confirm that getting this working in core is part of the plan?

davidebbo commented 7 years ago

@mattwoberts that's definitely a different topic, and is not directly related to Azure Functions. See https://stackoverflow.com/questions/38919723/how-do-i-deploy-an-azure-webjob-alongside-a-net-core-web-app-via-git#comment79729644_38919723 which discusses this.

piizei commented 7 years ago

Nuget dependencies fail with OSX. Does not help to install nuget mono version.

NuGet restore failed with message: 'Operation not permitted' [09/28/2017 08:31:51] Function directory: xx [09/28/2017 08:31:51] Project path: xx [09/28/2017 08:31:51] Packages path: /Users/pj/data\Functions\packages\nuget [09/28/2017 08:31:51] Nuget client path: nuget.exe [09/28/2017 08:31:51] Lock file hash: [09/28/2017 08:31:51] [09/28/2017 08:31:51] Package restore failed: [09/28/2017 08:31:51] [09/28/2017 08:31:51] System.ComponentModel.Win32Exception : Operation not permitted [09/28/2017 08:31:51] at Interop.Sys.ForkAndExecProcess(String filename,String[] argv,String[] envp,String cwd,Boolean redirectStdin,Boolean redirectStdout,Boolean redirectStderr,Int32& lpChildPid,Int32& stdinFd,Int32& stdoutFd,Int32& stderrFd) [09/28/2017 08:31:51] at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [09/28/2017 08:31:51] at System.Diagnostics.Process.Start()

ahmelsayed commented 7 years ago

what version of the cli do you have? can you run func help it should print the version, this is the current one 

Azure Functions Core Tools (2.0.0)
Function Runtime Version: 2.0.11280.0
fabiocav commented 7 years ago

There are known issues (we're documenting relevant issues here) with package restore that will be addressed with the next update, likely to occur by the end of next week.

fabiocav commented 7 years ago

Closing this as the initial port has been completed. New tasks tracking gaps, improvements and bug fixes will be open with the appropriate labels.

galvesribeiro commented 7 years ago

@fabiocav did everything got ported and out of preview?