Table support in Azurite V3

[asasine]

Which service(blob, file, queue, table) does this issue concern?

Table

Which version of the Azurite was used?

V3

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

All

What's the Node.js version?

N/A

What problem was encountered?

Table is not supported in Azurite V3

Steps to reproduce the issue?

N/A

Have you found a mitigation/solution?

Use Azurite V2

---

The Features Scope parapgraph of the README mentions Table service support is under discussion. Please add support for Table in Azurite V3. Microsoft's and Azure's strategy around Table storage has been murky and I would not like to see Table support dropped from Azurite.

[XiaoningLiu]

Thanks for your feedback!

There has been a discussion about table in V3. @edwin-huber may provide more backgrounds here. Edwin is also helping mataining table in Azurite V2.

Cosmos DB team is working an emulator for both CosmosDB & Table Storage. Please refer to here https://docs.microsoft.com/en-us/azure/cosmos-db/local-emulator

The Azure Storage Emulator is being deprecated in favor of Azurite V3. The Azure Storage Emulator will continue to be available for download via the standalone installer and via the Azure SDK and will continue to support the latest Storage REST service versions. New feature capabilities however, will be available only in Azurite starting REST service version 2019-02-02. Azure Table will not be supported in Azurite. Instead the CosmosDB emulator should be used for emulating Table access.

[ggirard07]

@XiaoningLiu where does theAzure storage emulator deprecation notice is coming from? I can't find it anywhere...

[XiaoningLiu]

@ggirard07 .Net based storage emulator will supports until storage API version 2019-02-02. Azurite will keeping updating and supporting new features of Azure Storage.

[ggirard07]

@XiaoningLiu thanks but this is still not an official statement :)

[asasine]

Cosmos DB emulator does not have support for Mac or Linux and requires you to run within a Windows VM. Even Docker on Mac or Linux does not work; only Docker for Windows.

Azurite is designed with cross-platform in mind and therefore works on Windows, Mac, and Linux without requiring special deployment; Cosmos emulator does not offer this. Requiring users to run a Windows VM to develop against a Table emulator is a blocker for the majority of developers.

[edwin-huber]

On this we agree :smile: , which is why we are maintaining support for Table Storage in v2 until Cosmos / Table Storage engineering team can release a cross platform emulator.
Unfortunately, in order to keep the work needed to achieve parity with the storage APIs under control, we had to re-architect Azurite.
This work cannot currently be replicated for Table storage, which is why we are not supporting Table storage in v3.
Please do keep issues, PRs and feedback coming.

[sleeping-barber]

@edwin-huber walking through the current CosmosDB documentation, make me believe there is still no Table Storage support and now cross platform solution. Is that correct? Thanks

[edwin-huber]

@midnightrun the table storage support is there, but we are still waiting on a cross platform version (Windows only atm):
https://docs.microsoft.com/en-us/azure/cosmos-db/local-emulator

[ggirard07]

@edwin-huber How is this intended to work when you need to consume both table and blob? In real world scenario, this is going to be the same endpoint offering both services. Whereas for local dev story, we will need to support having to configure 2 different endpoints, right?

[edwin-huber]

@ggirard07 - When we have a solution for both, I would hope that the port and authentication for local development will stay the same, thus not requiring any changes to address 2 endpoints, as it will be different processes on different ports, just as it is done now by Azurite starting multiple instances.
This was one of the many things that we have been looking at whilst creating a cross platform solution.
Unfortunately, I am not in a position to give you any insights into how we plan to solve table storage emulation at the moment, and have flagged it with the appropriate folks.

[MatejQ]

Isn't this issue duplicate of #428 (or vice versa)?

[macux]

I said to @jongio on twitter I would add a comment here and I don't currently have time to go back and check this so apologies if I'm out of date. I currently use the Azure storage emulator locally in development as well as in CI/CD (Azure DevOps) for running tests on agents. I use table storage and blobs in all of this.

When I last looked at moving to Azurite my issue I think was that the Azure storage emulator and Azurite supported different storage SDK versions so ultimately I had to stay on an older storage SDK and keep using storage emulator to do local dev + CI/CD with an emulator.

Basically I don't mind what emulator(s) I use so long as I can dev and test locally with emulator(s) that support both blobs and tables and that can also be used in Azure DevOps to run tests.

[edwin-huber]

Thanks all for the attention on this.
We are in the process of figuring this out, and soon hope to have a clear answer on our direction and a long term solution. @XiaoningLiu FYI.

[XiaoningLiu]

Hi guys, after serveral rounds of discussion and review. Azurite V3 is going to support Table. It's great if anyone likes to contribute to Azurite V3 Table. Use following queries for all opening TODO items:

Phase 1 Preview Features, All Phase 1 Preview Features, Unassigned Phase 1 Preview Testing, All Phase 1 Preview Testing, Unassigned Other references: Azurite V3 Table GitHub Branch: https://github.com/Azure/Azurite/tree/table Azurite V3 Table GitHub Project: https://github.com/Azure/Azurite/projects/2 Azurite V3 Table Wiki (under development): https://github.com/Azure/Azurite/wiki/Azurite-V3-Table

[jamesmcroft]

The work being done here to provide support for Table storage is incredible! Thanks everyone involved 👍

Looking through the milestones and the work already complete, what would the timescales be for the Phase 1 work to be available in some capacity for testing with?

Recently stumbled on this with our CI/CD failing builds do to Azurite not noticing that the issue was support not being there for Table storage.

[XiaoningLiu]

Here is for phase-1 features Azurite V3 Table going to support. https://github.com/azure/azurite/issues?q=is%3Aissue+label%3Anewarch+label%3Atable-storage+label%3Afeatureparity+milestone%3A%22Preview+-+Azurite+V3+Table%22+is%3Aopen

[sehcheese]

What is the status of this? Can we expect table storage support in Azurite V3 anytime soon?

[edwin-huber]

Hi, There are some behaviors which are not yet replicated, but please do take a look at the table branch, this is where we are putting in all of our effort to get the Table API ready for GA.
You are welcome to test and raise issues for problems that you see.
Thanks!

[johnnyreilly]

It looks like initial support for tables has landed:

https://github.com/Azure/Azurite/releases/tag/v3.12.0

@markheath has a nice post on it: https://markheath.net/post/azurite

[edwin-huber]

The VS Code extension will be updated closer to GA when we have closer feature parity (transaction rollback etc).

[johnnyreilly]

I found myself putting together an approach to work with Azurite in a dev container, such that you can make use of the Table Service APIs in VS Code before the new VS Code extension is released. If you'd like to see how to do that, I wrote it up here:

https://blog.johnnyreilly.com/2021/05/15/azurite-and-table-storage-dev-container

[jamesmcroft]

Thanks to everyone who has been working on progressing the Table storage work in Azurite. It looks like what is now available has allowed us to progress with our local dev and test setup for table storage 🙌🏻

[edwin-huber]

hi all, I'm closing this issue with the release of the API in v3 preview.