Does not work on Mac M1

meyntony commented 2 years ago

docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=<YourStrongPassw0rd#4b4b5c9e-cec8-49c1-8b50-a59e47ba4c63>" -p 56038:1433 --name umbracocloud-local-dev-sql-Modular-56038 -h Modular.umbracocloud-local-dev-sql -d mcr.microsoft.com/mssql/server:2019-latest
docker exec -it umbracocloud-local-dev-sql-Modular-56038 /opt/mssql-tools/bin/sqlcmd -S localhost -U SA -P "<YourStrongPassw0rd#4b4b5c9e-cec8-49c1-8b50-a59e47ba4c63>" -Q "CREATE DATABASE LocalDevUmbracoCloudModular"

The above code works fine on Windows, Ubuntu, Mac with Intel Chip, but not on Mac with the M1 chip.

Is this a known issue?

botanozdemir commented 1 year ago

@tracker1 just tried...

konstantinos-kafkas-cko commented 1 year ago

Guys we need this urgently, please follow up

dkuhta commented 1 year ago

M1 support needed urgently!

rubnogueira commented 1 year ago

This is really really import for development. However, I think some progress is being made here: https://github.com/docker/roadmap/issues/384

yori-s commented 1 year ago

This has come up in other comments, but I would like chime in that Azure SQL Edge was not an adequate substitute in my case, which is making an existing dockerized development environment availble to M1 users:

full-text search - minimally used, so stripped it out for the purposes of testing edge
CLR support - failed over 50% of ~1200 integration tests, AT TIME ZONE being a major culprit
'Mounting a volume does not work on Docker for Mac' - aside from the inconvenience, this is my first encounter with an image that doesn't support volumes, and runs counter to the purpose of the exercise

It would be nice to get an authoritative - yes we'll support M1 (when) or no, not going to happen

AndreaGernone commented 1 year ago

Hi everyone! If you're still having issues with the latest image and this command:

docker run --platform linux/amd64 --name SQLServer -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD={YOUR_PASSWORD}*AL' -e 'MSSQL_PID=Developer' -p 1433:1433 mcr.microsoft.com/azure-sql-edge:latest

I personally solved it by using a container instead of the run command. The reason is that they pull 2 different images and the one pulled from the yml file is the working one on M1 Mac.

So, simply create a docker-compose.yml file with this inside: ` version: "3" services: mssql: image: mcr.microsoft.com/azure-sql-edge:latest user: root environment:

MSSQL_PID=Developer
MSSQL_SA_PASSWORD={YOUR_PASSWORD}
ACCEPT_EULA=Y volumes:
mssql:/var/opt/mssql/data ports:
"1433:1433" volumes: mssql: `

And then use the command docker-compose up . It will pull the right image and start running with no problems. Cheers

tracker1 commented 1 year ago

'Mounting a volume does not work on Docker for Mac' - aside from the inconvenience, this is my first encounter with an image that doesn't support volumes, and runs counter to the purpose of the exercise

Haven't had a problem mounting in a mac... I will say that mount performance is problematic in mac and windows directories, and can be slow to synchronize and may experience timing issues in practice. Best to use a volume container and reference that. Can still copy in/out as needed. I do mount directly for backup directory for backup/restore command operations.

It would be nice to get an authoritative - yes we'll support M1 (when) or no, not going to happen

I wouldn't hold my breath... honestly, may see fixes come from apple and docker for the underlying issues before a fix from MS for this. Might be more likely to get the missing features you need added to the azure-sql-edge service.

TheMuyu commented 1 year ago

Apple M1 Silicon support needed urgently!

osamahq commented 1 year ago

M1 support is needed urgently

samuelkitazume commented 1 year ago

M1 support is needed urgently

kralle333 commented 1 year ago

M1 support is needed urgently

wcoder commented 1 year ago

M1 support is needed urgently

amasnik commented 1 year ago

M1 support is needed urgently

devanilish commented 1 year ago

Please add support for Apple M1 silicon urgently!!

jclausen commented 1 year ago

Since M1 is now the only available processor arch for MacBook Pro machines, as developers upgrade their hardware, they lose the ability to use the SQL Server image.

The Azure Edge image works for basic functionality against SQL server, but does not include full text, spatial functions, nor simple conversion functions like cast( myColumnName as "hierarchyId").

With the availability of Docker BuildX, I fail to see why building to support arm64 is problematic, and have a hard time believing the SQL Server for Linux team can't figure out how to handle ARM architecture to enable this feature set. The issues around ARM support are only going to continue to magnify in the months and years to come.

newGeneracion commented 1 year ago

This worked for me, my dockerfile (special attention to freetds-bin freetds-dev)

`FROM python:3.10.0

ENV ENV local ENV PYTHONDONTWRITEBYTECODE=1 ENV PYTHONUNBUFFERED=1

RUN apt update -y

RUN apt install -y libxml2-dev libxmlsec1-dev libxmlsec1-openssl xmlsec1 libxslt1-dev \ xvfb libfontconfig python3-dev build-essential wkhtmltopdf gettext freetds-bin freetds-dev

RUN pip install pip==22.3.1

RUN mkdir -p /voldemort_app/tmp

COPY ./requirements.txt /voldemort_app/requirements.txt

RUN pip install --no-cache-dir -r /voldemort_app/requirements.txt \ --extra-index-url=https://pypi.viewflow.io/12F7DABD8D91/simple/

COPY . /voldemort_app

RUN adduser --disabled-password --gecos '' myuser

RUN chmod +x /voldemort_app/docker-compose//entrypoint.sh \ && chmod +x /voldemort_app/docker-compose/wait-for-it.sh

WORKDIR /voldemort_app

EXPOSE 5000`

hoho00 commented 1 year ago

M1 support is needed urgently

tracker1 commented 1 year ago

https://github.com/docker/roadmap/issues/384

jclausen commented 1 year ago

docker/roadmap#384

I've seen that. It's not an optimal solution as the Rosetta 2 emulation is significantly worse in performance. The solution is just to use Docker BuildX to build multi-arch images, like so. The Azure Edge images are doing this already.

tracker1 commented 1 year ago

I've seen that. It's not an optimal solution as the Rosetta 2 emulation is significantly worse in performance. The solution is just to use Docker BuildX to build multi-arch containers, like so. The Azure Edge containers are doing this already.

Where Rosetta 2 is available, it's not that much worse and significantly better than the QEMU emulation being used for x86_64 currently.

While a native aarch64 build of SQL Server would be better still, I don't know that it's even on Microsoft's radar, as a single ARM core is unlikely to work well, and the Azure Edge containers, as you mention already do this. Asking for the missing features in Edge version are probably more likely to see response sooner than getting a full SQL Server on Arm release at all.

thisisthekap commented 1 year ago

I can confirm, that running SQL Server with Rosetta 2 works like a charm.

What I did to get it working using colima:

brew install --HEAD lima

git clone https://github.com/abiosoft/colima.git
cd colima
git checkout support-vz
make build
sudo make install

# set driver explicitly to vz in this step
colima template

# --network-address is needed to have data transfer to containers working as expected
# number of cpus and amount of GBs RAM may be defined according to your own needs
colima start --network-address --cpu 8 --memory 8

This script is based of the work of @mmeyer724 (https://github.com/docker/roadmap/issues/384#issuecomment-1329712712).

Make sure to run docker context list to check which docker context is used (the new colima context needs to be selected).

After that, you may just run docker run -e ACCEPT_EULA=Y mcr.microsoft.com/mssql/server:2019-CU18-ubuntu-20.04

Edit: Fixed the script as pointed out by @Will-Bill in https://github.com/microsoft/mssql-docker/issues/734#issuecomment-1333729105

Will-Bill commented 1 year ago

Above is slightly incorrect, need to clone colima.

brew install --HEAD lima

git clone https://github.com/abiosoft/colima.git && cd colima
git checkout support-vz
make build
sudo make install

colima start

adam-at-mobomo commented 1 year ago

Thanks, @thisisthekap and @Will-Bill. The above almost worked for me, but I am not trying to run SQL Server. Instead, I need to run a container and also do a volume mount, and that did not work. However, if you make one more change, it does:

# set driver explicitly to vz and also set the mountType to virtiofs in this step
colima template

invaderb commented 1 year ago

Tried the steps above and is works on a m1 pro with colima but on a m1 max it returns a warning and nothing actually loads The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64)

adam-at-mobomo commented 1 year ago

Tries the steps above and is works on a m1 pro with colima but on a m1 max it returns a warning and nothing actually loads The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64)

@invaderb, I'm not sure why Max would be any different than Pro. I assume you are getting this when trying to run a docker image. Try adding --platform linux/amd64 like this:

docker run --rm --platform linux/amd64 busybox arch

(for whatever x86 image you are trying to run)

invaderb commented 1 year ago

@adam-at-mobomo Yeah I was adding it to the docker file FROM --platform=linux/amd64 php:7.3-apache-buster but I needed to add it to the docker compose yml file as well which resolved the warning but having some other volume sync issue now but sure it's not related to this.

overengineeredit commented 1 year ago

I tried the suggest steps to use colima with container mcr.microsoft.com/mssql/server:2017-latest to no effect. Am still getting the same error as the initially noted one:


/opt/mssql/bin/sqlservr: Invalid mapping of address 0x40094e5000 in reserved address space below 0x400000000000. Possible causes:
1) the process (itself, or via a wrapper) starts-up its own running environment sets the stack size limit to unlimited via syscall setrlimit(2);
2) the process (itself, or via a wrapper) adjusts its own execution domain and flag the system its legacy personality via syscall personality(2);
3) sysadmin deliberately sets the system to run on legacy VA layout mode by adjusting a sysctl knob vm.legacy_va_layout.```

invaderb commented 1 year ago

I tried the suggest steps to use colima with container mcr.microsoft.com/mssql/server:2017-latest to no effect. Am still getting the same error as the initially noted one:



/opt/mssql/bin/sqlservr: Invalid mapping of address 0x40094e5000 in reserved address space below 0x400000000000. Possible causes:

1) the process (itself, or via a wrapper) starts-up its own running environment sets the stack size limit to unlimited via syscall setrlimit(2);

2) the process (itself, or via a wrapper) adjusts its own execution domain and flag the system its legacy personality via syscall personality(2);

3) sysadmin deliberately sets the system to run on legacy VA layout mode by adjusting a sysctl knob vm.legacy_va_layout.```

are you sure your using the Colima runtime?

run "docker context list" and see what it's set too

sinfergit commented 1 year ago

I tried the suggest steps to use colima with container mcr.microsoft.com/mssql/server:2017-latest to no effect. Am still getting the same error as the initially noted one:
/opt/mssql/bin/sqlservr: Invalid mapping of address 0x40094e5000 in reserved address space below 0x400000000000. Possible causes:

1) the process (itself, or via a wrapper) starts-up its own running environment sets the stack size limit to unlimited via syscall setrlimit(2);

2) the process (itself, or via a wrapper) adjusts its own execution domain and flag the system its legacy personality via syscall personality(2);

3) sysadmin deliberately sets the system to run on legacy VA layout mode by adjusting a sysctl knob vm.legacy_va_layout.```
are you sure your using the Colima runtime?

run "docker context list" and see what it's set too

I got the same issue and checked the docker context as in the below image. Colima has been selected as it should be.

Any idea as to what could be the reason?

darrinholst commented 1 year ago

I tried the suggest steps to use colima with container mcr.microsoft.com/mssql/server:2017-latest to no effect. Am still getting the same error as the initially noted one:
/opt/mssql/bin/sqlservr: Invalid mapping of address 0x40094e5000 in reserved address space below 0x400000000000. Possible causes:

1) the process (itself, or via a wrapper) starts-up its own running environment sets the stack size limit to unlimited via syscall setrlimit(2);

2) the process (itself, or via a wrapper) adjusts its own execution domain and flag the system its legacy personality via syscall personality(2);

3) sysadmin deliberately sets the system to run on legacy VA layout mode by adjusting a sysctl knob vm.legacy_va_layout.```
are you sure your using the Colima runtime? run "docker context list" and see what it's set too
I got the same issue and checked the docker context as in the below image. Colima has been selected as it should be.

Any idea as to what could be the reason?

Run colima status to make sure it’s using the new virtualization framework and not qemu still.

sinfergit commented 1 year ago

I tried the suggest steps to use colima with container mcr.microsoft.com/mssql/server:2017-latest to no effect. Am still getting the same error as the initially noted one:
/opt/mssql/bin/sqlservr: Invalid mapping of address 0x40094e5000 in reserved address space below 0x400000000000. Possible causes:

1) the process (itself, or via a wrapper) starts-up its own running environment sets the stack size limit to unlimited via syscall setrlimit(2);

2) the process (itself, or via a wrapper) adjusts its own execution domain and flag the system its legacy personality via syscall personality(2);

3) sysadmin deliberately sets the system to run on legacy VA layout mode by adjusting a sysctl knob vm.legacy_va_layout.```
are you sure your using the Colima runtime? run "docker context list" and see what it's set too
I got the same issue and checked the docker context as in the below image. Colima has been selected as it should be. Any idea as to what could be the reason?
Run colima status to make sure it’s using the new virtualization framework and not qemu still.

For some reason when I ran colima status it gives me an error saying FATA[0000] colima is not running but when I run the command to start Colima it runs without any issue. Also I noticed that vmType in Colima Template was as qemu and I manually changed it to vz but still the same error occurs

invaderb commented 1 year ago

@sinfergit @darrinholst did you get any errors when you installed it? I'd suggest clearing everything out and re-installing lima and colima by following the steps above it sounds like something is borked.

sinfergit commented 1 year ago

@sinfergit @darrinholst did you get any errors when you installed it? I'd suggest clearing everything out and re-installing lima and colima by following the steps above it sounds like something is borked.

The only error I got running the script is from here error: pathspec 'support-vz' did not match any file(s) known to git And when I checked there is no such branch in Colima Git repository and I checked out to main branch.

Did everything from the scratch again and now getting the following error about the colima driver.

darrinholst commented 1 year ago

You shouldn't have to build anything from source anymore. Here is what I just did to clean and reinstall...

brew uninstall --force colima lima
rm -rf ~/.colima ~/.lima
brew install colima
colima template

Change vmType to vz and mountType to virtiofs. I also adjust CPU, disk, and memory here so I don't have to add them to the start command.

colima start
colima status

INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] socket: unix:///Users/darrinholst/.colima/default/docker.sock

docker run --rm -e ACCEPT_EULA=Y mcr.microsoft.com/mssql/server:2019-CU18-ubuntu-20.04

2022-12-31 13:47:53.57 spid24s     Server is listening on [ 'any' <ipv6> 1433].
2022-12-31 13:47:53.58 spid24s     Server is listening on [ 'any' <ipv4> 1433].
2022-12-31 13:47:53.59 Server      Server is listening on [ ::1 <ipv6> 1434].
2022-12-31 13:47:53.59 Server      Server is listening on [ 127.0.0.1 <ipv4> 1434].
2022-12-31 13:47:53.59 Server      Dedicated admin connection support was established for listening locally on port 1434.
2022-12-31 13:47:53.59 spid24s     Server is listening on [ ::1 <ipv6> 1431].
2022-12-31 13:47:53.59 spid24s     Server is listening on [ 127.0.0.1 <ipv4> 1431].
2022-12-31 13:47:53.60 spid24s     SQL Server is now ready for client connections. This is an informational message; no user action is required.

sinfergit commented 1 year ago

You shouldn't have to build anything from source anymore. Here is what I just did to clean and reinstall...

brew uninstall --force colima lima
rm -rf ~/.colima ~/.lima
brew install colima
colima template

Change vmType to vz and mountType to virtiofs. I also adjust CPU, disk, and memory here so I don't have to add them to the start command.

colima start
colima status

INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] socket: unix:///Users/darrinholst/.colima/default/docker.sock

docker run --rm -e ACCEPT_EULA=Y mcr.microsoft.com/mssql/server:2019-CU18-ubuntu-20.04

2022-12-31 13:47:53.57 spid24s     Server is listening on [ 'any' <ipv6> 1433].
2022-12-31 13:47:53.58 spid24s     Server is listening on [ 'any' <ipv4> 1433].
2022-12-31 13:47:53.59 Server      Server is listening on [ ::1 <ipv6> 1434].
2022-12-31 13:47:53.59 Server      Server is listening on [ 127.0.0.1 <ipv4> 1434].
2022-12-31 13:47:53.59 Server      Dedicated admin connection support was established for listening locally on port 1434.
2022-12-31 13:47:53.59 spid24s     Server is listening on [ ::1 <ipv6> 1431].
2022-12-31 13:47:53.59 spid24s     Server is listening on [ 127.0.0.1 <ipv4> 1431].
2022-12-31 13:47:53.60 spid24s     SQL Server is now ready for client connections. This is an informational message; no user action is required.

Thanks a lot @darrinholst now it works. So what was missing was the mountType to be updated to virtiofs. Thanks for you time 👍

Meligy commented 1 year ago

What's the point of changing Colima defaults for SQL Server? I have been running it successfully for a year without changing anything.

Is it faster or anything?

I also don't share volumes with the host. Is that what's broken by the defaults?

invaderb commented 1 year ago

What's the point of changing Colima defaults for SQL Server? I have been running it successfully for a year without changing anything.

Is it faster or anything?

I also don't share volumes with the host. Is that what's broken by the defaults?

@Meligy the issue specifically steams for m1 macs as there isn't a native mssql container. if your on an Intel Mac or windows there isn't a strong reason to switch.

Meligy commented 1 year ago

@Meligy the issue specifically steams for m1 macs as there isn't a native mssql container

@invaderb, yes. I get that. I have an M1 Max MacbBook Pro 16". And I use colima for that.

Sorry if my question was not clear but I was asking about the reason for customising the colima template vs using colima's defaults, for someone who's already using colima to run SQL Server on Mac.

darrinholst commented 1 year ago

Is it faster or anything?

Yes. Significantly. For some people just getting it to run with qemu emulation on arm64 was good enough. For others the performance was a show stopper.

In my case we’re using it for automated testing against a local database (with the need for the CLR so the azure image was out). It’s the difference between seconds and literally minutes.

Meligy commented 1 year ago

OK. The new settings in colima ARE cool, much faster indeed.

I started by deleting the existing VM then following the cleanup instructions above (my default template was not including the vmType setting without them):

colima delete
brew uninstall --force colima lima
rm -rf ~/.colima ~/.lima
brew install colima

Then I applied the settings to colima template:

vmType: vz
mountType: virtiofs

I forgot to measure the colima start command time difference, it certainly felt faster, but I could see the speeds of starting a custom SQL Server image (for enabling Full Text Search etc.) changing quite a bit.

First, I want to mention how I changed my start command, because it was problematic. I changed it as follows...

From (OLD):

colima start --arch x86_64 --cpu-type kvm64 --cpu 4 --memory 4 --disk 60

To (NEW):

colima start --network-address --cpu 4 --memory 4 --disk 60

Looking at the default comments in the template, I think at least setting cpu-type was reverting my change to QEMU.

And the results were amazing.

Results BEFORE (I cancelled when I sensed nothing changed from usual):

Results AFTER (Look at same-shape arrows for comparison):

The download time of the 784.49MB part ending in f3a3464f went down from 64.3s to 48.8s, but the most amazing time difference was the extraction of the same part, going down from 211.2s to just 12.4s!! Not to mention that also my custom apt-get commands time went from 142.0s to 45.6s.

That's going from minutes to seconds as @darrinholst said!

Thanks for sharing!

And hopefully at some point people won't have to go through all of this.

jordantshaw commented 1 year ago

UPDATE: The new docker-desktop release (v4.16.0) introduces a new Mac beta feature, Rosetta for Linux, which helps performance and stability when running intel bases images. I am now able to run MSSQL docker image on my Mac M1 laptop with no issues yet. Also seems pretty performant, but have not run any benchmarks yet.

It would still be better if microsoft would provide a native ARM image, but this seems to be the best work around for now.

Release Notes: New Beta feature for MacOS 13, Rosetta for Linux, has been added for faster emulation of Intel-based images on Apple Silicon.

dzsquared commented 1 year ago

Recapping where arm64 (M1/M2) support is currently for SQL Server:

to run the mssql container, enable Rosetta for emulation in Docker Desktop v4.16 or higher (also requires Ventura) https://devblogs.microsoft.com/azure-sql/development-with-sql-in-containers-on-macos/
- alternatively, you can setup Colima
- also alternatively, you can use Azure SQL Edge
Azure Data Studio has an arm64 macOS build https://github.com/microsoft/azuredatastudio
- alternatively, you can use the mssql extension in VS Code
go-sqlcmd cli has native arm64 builds available in brew and for direct download https://docs.microsoft.com/sql/tools/go-sqlcmd-utility

if I've missed anything from the 91 comments in this thread, please feel free to open a new issue on that specific item!

glerchundi commented 1 year ago

@dzsquared thanks for the list! Would you mind adding to that list that macOS Ventura (13) is required? I've gone crazy looking for that checkbox within the Docker Desktop and realised that it wasn't present because I was on 12.

Meligy commented 1 year ago

I don't know about the need for latest Mac, but there's another gotcha to be aware of that I mentioned here: https://github.com/docker/roadmap/issues/384#issuecomment-1382615753.

In short: there are 2 beta features in the same tab, the Rosetta emulation and containerd for image pulling and storage. The containerd feature is supposed to be much faster but in my experience it caused an error, and after turning it off, the image pulling and extraction was still fairly fast (71.7s vs 61.3s on colima as mentioned above).

Jolg42 commented 1 year ago

Someone confirmed macOS Ventura is required, see https://github.com/docker/roadmap/issues/384#issuecomment-1385037769, I'll upgrade soon to get this on my computer. It's also mentioned in https://devblogs.microsoft.com/azure-sql/development-with-sql-in-containers-on-macos/

In the v4.16 release of Docker Desktop, support for using Rosetta for emulation of x86/amd64 images was introduced as a beta feature, enabling improved performance and stability for running container images built for x86/amd64 on macOS 13 (Ventura).

tinydogio-joshua commented 1 year ago

Are there any plans for an official ARM container? My understanding is that Rosetta will most likely go away at some point. It would be nice to see an official ARM build.

theramzay commented 1 year ago

+1 for native arm build, because Rosetta 2 like Rosetta 1 -- just temporary workaround for 2-4 years.

mkurz commented 1 year ago

Yeah, that would be really great! aarch64 build now :wink:

jclausen commented 1 year ago

I can confirm that running the x86_64 SQL Server container under Rosetta is not an ideal solution. Due to the arch emulation, the SQL Server container under load locks up and becomes unresponsive. My M1-using devs report they are constantly having to restart the SQL Server container. They can use it, but it's buggy.

mkurz commented 1 year ago

Yeah it sucks! Give us the real thing :pray:

microsoft / mssql-docker

Does not work on Mac M1 #734