When watchInitializationJson is enabled, updating expectations through expectation ID will clear all expectations

maojunxyz commented 3 years ago

Describe the issue When watchInitializationJson is enabled, updating expectations through expectation ID will clear all expectations

What you are trying to do

        ConfigurationProperties.persistExpectations(true);
        ConfigurationProperties.persistedExpectationsPath("initialJson.json");
        ConfigurationProperties.initializationJsonPath("initialJson.json");
        ConfigurationProperties.watchInitializationJson(true);
        ClientAndServer mockServer = startClientAndServer(8011);

MockServer version 5.11.2

To Reproduce Steps to reproduce the issue:

How you are running MockServer (i.e maven plugin, docker, etc) maven dependency
Code you used to create expectations

 JSONObject jsonObject = new JSONObject();
        String str = JSON.toJSONString(createJson);
        jsonObject.put("reqBody", str);
        CloseableHttpResponse response = doPut("http://127.0.0.1:8011/mockserver/expectation", jsonObject);

What error you saw Concurrent updating expectations through the same expectation ID，Will Always clear all expectations

Expected behaviour Concurrent updating expectations through same expectation ID,Just updat By Id.

MockServer Log

08-05 16:25:34.128 CREATED_EXPECTATION      creating expectation:   
{...}
with id:    "02ccf63d-efaa-4531-bbf4-e4d48d28ddc1"
08-05 16:25:33.799 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "02ccf63d-efaa-4531-bbf4-e4d48d28ddc1"
08-05 16:25:33.793 CREATED_EXPECTATION      creating expectation:   
{...}
with id:    "02ccf63d-efaa-4531-bbf4-e4d48d28ddc1"
08-05 16:25:33.586 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "2bfaa97c-d057-4b95-8a66-e37dc1fde079"
08-05 16:25:33.583 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "c17cb62d-66c8-446f-a136-9abfdee0b00a"
08-05 16:25:33.583 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "e948493e-3fe8-40e8-a813-22b5d2bedb42"
08-05 16:25:33.583 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "ab149fb3-bf21-46b7-9689-68b08f8ca1d2"
08-05 16:25:33.583 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "289c421a-6431-4aa7-b981-fde201454d54"
08-05 16:25:33.583 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "75d746e6-7fea-4f5f-9ac6-3760561801a3"
08-05 16:25:33.583 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "e4375714-0485-4110-beb8-ba3cb84c2a1c"
08-05 16:25:33.583 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "d38e7074-f0ad-4f6b-b3c3-909096c9c38d"
08-05 16:25:33.583 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "f449466e-5210-4077-8f66-e9b16d05bc8e"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "f3447067-7e6d-4007-a287-b7a99d04d9ba"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "cf2c751f-638b-4211-b9c8-7fc4d8716f77"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "46eb1919-64f1-41d8-891b-0e859b00eb3f"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "b1891e8e-5b0f-47a8-a848-4c0fa831edaa"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "4bc9be70-c5d6-477f-8af6-f7d84a69271a"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "28157c3a-5ec1-43c0-a92a-412e80d6a170"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "47354de6-6ba1-4250-8f00-39bcb37c6fd1"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "f9268b2b-b003-497b-a4bd-aaab7c03f276"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "80e99546-051a-4b76-b51c-0faeb059a1aa"
08-05 16:25:33.582 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "43409f76-2231-46fb-9cf5-62ab0d433c3b"
08-05 16:25:33.579 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "a8708bf4-5998-4292-9a9b-7120ecc2dd57"
08-05 16:25:33.579 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "7ea810f7-f746-40fe-a75f-428805e3886c"
08-05 16:25:33.579 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "69ffea28-353f-4865-8d61-fab86d40df24"
08-05 16:25:33.579 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "ba9b4896-99fb-4745-9866-da000a7cfb76"
08-05 16:25:33.579 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "438f2724-f2c8-4b85-8061-ea7e58986db4"
08-05 16:25:33.579 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "666f9018-6d10-4a21-8634-1365d55af565"
08-05 16:25:33.579 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "7eaaa084-8265-4c6a-ad8b-d31a0b94ad4c"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "d43e2bfa-cdda-4968-98aa-6e87c8575e75"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "eb9db72a-1a8e-4617-86fe-c8c021a6d876"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "e82309c9-94c1-4546-8b79-44bd7244d70f"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "96a671e8-3788-4a08-a403-8c2912ccb2ac"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "e3ff01eb-e052-4de9-9b00-bb00e92fde81"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "7c448856-b542-4482-8038-f29ba09d65dd"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "0b60eb68-95b1-4cc2-b73f-804a95c46586"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "a44800db-5af7-4be2-b082-bf08f31df46c"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "df900575-233f-44bd-8766-777d485f467a"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "02ccf63d-efaa-4531-bbf4-e4d48d28ddc1"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "9d9259e1-1be2-4786-9320-098a3f9daefc"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "1a67cc67-6198-4777-a68a-af8106dfaf0b"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "9ff8b8bc-4e3b-469f-b1f3-ef19e18d4ec8"
08-05 16:25:33.574 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "88bd8d46-cab2-4e2f-8bdc-ed3f791c346e"
08-05 16:25:33.566 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "557df5ae-c2c3-405a-95bb-ca407cef2998"
08-05 16:25:33.566 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "4586d72c-b274-48b7-b693-c136ec862a7a"
08-05 16:25:33.566 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "eec77d65-07e2-4d55-bf45-9ff8f8a9158c"
08-05 16:25:33.566 REMOVED_EXPECTATION      removed expectation:    
{...}
with id:    "1453cb26-4b4b-4ed2-be1e-07a4420a6cae"
08-05 16:25:33.344 UPDATED_EXPECTATION      updated expectation:    
{...}
with id:    "02ccf63d-efaa-4531-bbf4-e4d48d28ddc1"
08-05 16:24:56.680 UPDATED_EXPECTATION      updated expectation:    
{...}
with id:    "02ccf63d-efaa-4531-bbf4-e4d48d28ddc1"

In addition, I saw the demo shows that the stand-alone configuration is somewhat different from the cluster configuration. Clustering MockServer openwatchInitializationJson=true, Won't the above problems occur in the cluster environment?

-Dmockserver.watchInitializationJson=true \
-Dmockserver.initializationJsonPath=mockserverInitialization.json \
-jar ~/Downloads/mockserver-netty-5.11.1-jar-with-dependencies.jar -serverPort 1080,1081 -logLevel INFO

Clustering MockServer

-Dmockserver.watchInitializationJson=true \
-Dmockserver.initializationJsonPath=mockserverInitialization.json \
-Dmockserver.persistExpectations=true \
-Dmockserver.persistedExpectationsPath=mockserverInitialization.json \
-jar ~/Downloads/mockserver-netty-5.11.1-jar-with-dependencies.jar -serverPort 1080 -logLevel INFO

adnan-vialogic commented 2 years ago

dockerfiles-mockServer-1 | 2022-03-02 10:34:03 5.12.0 SEVERE exception creating persisted expectations file mockserverInitialization.json dockerfiles-mockServer-1 | java.nio.file.AccessDeniedException: mockserverInitialization.json dockerfiles-mockServer-1 | at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90) dockerfiles-mockServer-1 | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) dockerfiles-mockServer-1 | at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) dockerfiles-mockServer-1 | at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) dockerfiles-mockServer-1 | at java.base/java.nio.file.Files.newByteChannel(Files.java:371) dockerfiles-mockServer-1 | at java.base/java.nio.file.Files.createFile(Files.java:648) dockerfiles-mockServer-1 | at org.mockserver.persistence.ExpectationFileSystemPersistence.(ExpectationFileSystemPersistence.java:47) dockerfiles-mockServer-1 | at org.mockserver.mock.HttpState.(HttpState.java:116) dockerfiles-mockServer-1 | at org.mockserver.lifecycle.LifeCycle.(LifeCycle.java:52) dockerfiles-mockServer-1 | at org.mockserver.netty.MockServer.(MockServer.java:97) dockerfiles-mockServer-1 | at org.mockserver.netty.MockServer.(MockServer.java:76) dockerfiles-mockServer-1 | at org.mockserver.cli.Main.main(Main.java:146) dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | 2022-03-02 10:34:03 5.12.0 INFO created expectation file system persistence for dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | mockserverInitialization.json dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | 2022-03-02 10:34:03 5.12.0 INFO watching file: dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | /./mockserverInitialization.json dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | with file fingerprint: dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | 0 dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | 2022-03-02 10:34:03 5.12.0 INFO created expectation file watcher for dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | [mockserverInitialization.json] dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | 2022-03-02 10:34:03 5.12.0 INFO loading JSON initialization file: dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | mockserverInitialization.json dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | 2022-03-02 10:34:03 5.12.0 WARNING exception while loading JSON initialization file, ignoring file: dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | mockserverInitialization.json dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | java.lang.RuntimeException: Exception while loading "mockserverInitialization.json" dockerfiles-mockServer-1 | at org.mockserver.file.FileReader.readFileFromClassPathOrPath(FileReader.java:38) dockerfiles-mockServer-1 | at org.mockserver.server.initialize.ExpectationInitializerLoader.lambda$retrieveExpectationsFromJson$0(ExpectationInitializerLoader.java:126) dockerfiles-mockServer-1 | at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:271) dockerfiles-mockServer-1 | at java.base/java.util.Collections$2.tryAdvance(Collections.java:4747) dockerfiles-mockServer-1 | at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4755) dockerfiles-mockServer-1 | at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) dockerfiles-mockServer-1 | at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) dockerfiles-mockServer-1 | at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) dockerfiles-mockServer-1 | at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) dockerfiles-mockServer-1 | at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) dockerfiles-mockServer-1 | at org.mockserver.server.initialize.ExpectationInitializerLoader.retrieveExpectationsFromJson(ExpectationInitializerLoader.java:156) dockerfiles-mockServer-1 | at org.mockserver.server.initialize.ExpectationInitializerLoader.addExpectationsFromInitializer(ExpectationInitializerLoader.java:53) dockerfiles-mockServer-1 | at org.mockserver.server.initialize.ExpectationInitializerLoader.(ExpectationInitializerLoader.java:40) dockerfiles-mockServer-1 | at org.mockserver.mock.HttpState.(HttpState.java:122) dockerfiles-mockServer-1 | at org.mockserver.lifecycle.LifeCycle.(LifeCycle.java:52) dockerfiles-mockServer-1 | at org.mockserver.netty.MockServer.(MockServer.java:97) dockerfiles-mockServer-1 | at org.mockserver.netty.MockServer.(MockServer.java:76) dockerfiles-mockServer-1 | at org.mockserver.cli.Main.main(Main.java:146) dockerfiles-mockServer-1 | Caused by: java.io.FileNotFoundException: mockserverInitialization.json (No such file or directory) dockerfiles-mockServer-1 | at java.base/java.io.FileInputStream.open0(Native Method) dockerfiles-mockServer-1 | at java.base/java.io.FileInputStream.open(FileInputStream.java:219) dockerfiles-mockServer-1 | at java.base/java.io.FileInputStream.(FileInputStream.java:157) dockerfiles-mockServer-1 | at java.base/java.io.FileInputStream.(FileInputStream.java:112) dockerfiles-mockServer-1 | at org.mockserver.file.FileReader.openStreamToFileFromClassPathOrPath(FileReader.java:46) dockerfiles-mockServer-1 | at org.mockserver.file.FileReader.readFileFromClassPathOrPath(FileReader.java:35) dockerfiles-mockServer-1 | ... 17 more dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | 2022-03-02 10:34:04 5.12.0 INFO using proxy configuration for forwarded requests: dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | [] dockerfiles-mockServer-1 | dockerfiles-mockServer-1 | 2022-03-02 10:34:04 5.12.0 FINE logger level is DEBUG, change using: dockerfiles-mockServer-1 | - 'ConfigurationProperties.logLevel(String level)' in Java code, dockerfiles-mockServer-1 | - '-logLevel' command line argument, dockerfiles-mockServer-1 | - 'mockserver.logLevel' JVM system property or, dockerfiles-mockServer-1 | - 'mockserver.logLevel' property value in 'mockserver.properties' dockerfiles-mockServer-1 | 2022-03-02 10:34:04 5.12.0 INFO 1090 started on port: 1090

My docker compose file

version: "2.4" services: mockServer: image: mockserver/mockserver:mockserver-5.12.0 command: -logLevel DEBUG -serverPort 1090 -proxyRemotePort 80 -proxyRemoteHost www.mock-server.com ports:

1080:1090 environment: MOCKSERVER_LOG_LEVEL: TRACE MOCKSERVER_MAX_EXPECTATIONS: 100 MOCKSERVER_MAX_HEADER_SIZE: 8192 MOCKSERVER_PERSIST_EXPECTATIONS: "true" MOCKSERVER_WATCH_INITIALIZATION_JSON: "true" MOCKSERVER_ENABLE_CORS_FOR_API: "true" MOCKSERVER_ENABLE_CORS_FOR_ALL_RESPONSES: "true" MOCKSERVER_CORS_ALLOW_CREDENTIALS: "true" MOCKSERVER_CORS_ALLOW_METHODS: "CONNECT, DELETE, GET, HEAD, OPTIONS, POST, PUT, PATCH, TRACE" MOCKSERVER_CORS_ALLOW_HEADERS: "Allow, Content-Encoding, Content-Length, Content-Type, ETag, Expires, Last-Modified, Location, Server, Vary, Authorization" MOCKSERVER_INITIALIZATION_JSON_PATH: mockserverInitialization.json MOCKSERVER_PERSISTED_EXPECTATIONS_PATH: mockserverInitialization.json volumes:
- ./config/:/config

AlexRuiz7 commented 2 years ago

Same trouble here. I mapped a a config directory to the /config folder in the container, enabled expectations persistence and defined the path, as seen below.

volumes:
  - ./config:/config
environment:
  MOCKSERVER_LOG_LEVEL: INFO
  MOCKSERVER_PERSIST_EXPECTATIONS: "true"
  MOCKSERVER_PERSISTED_EXPECTATIONS_PATH: /config/modified expectations.json

This throws the following exception (permissions):

mock-server_1  | 2022-03-18 14:15:13 5.13.0 SEVERE exception creating persisted expectations file /config/modified expectations.json 
mock-server_1  | java.nio.file.AccessDeniedException: /config/modified expectations.json
mock-server_1  |        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
mock-server_1  |        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
mock-server_1  |        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
mock-server_1  |        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
mock-server_1  |        at java.base/java.nio.file.Files.newByteChannel(Files.java:371)
mock-server_1  |        at java.base/java.nio.file.Files.createFile(Files.java:648)
mock-server_1  |        at org.mockserver.persistence.ExpectationFileSystemPersistence.<init>(ExpectationFileSystemPersistence.java:49)
mock-server_1  |        at org.mockserver.mock.HttpState.<init>(HttpState.java:122)
mock-server_1  |        at org.mockserver.lifecycle.LifeCycle.<init>(LifeCycle.java:55)
mock-server_1  |        at org.mockserver.netty.MockServer.<init>(MockServer.java:77)
mock-server_1  |        at org.mockserver.netty.MockServer.<init>(MockServer.java:48)
mock-server_1  |        at org.mockserver.cli.Main.main(Main.java:145)

I have also created the specified file in the folder with touch but the problem persists.

y00ss commented 1 month ago

Same trouble here. I mapped a a config directory to the /config folder in the container, enabled expectations persistence and defined the path, as seen below.

volumes:
  - ./config:/config
environment:
  MOCKSERVER_LOG_LEVEL: INFO
  MOCKSERVER_PERSIST_EXPECTATIONS: "true"
  MOCKSERVER_PERSISTED_EXPECTATIONS_PATH: /config/modified expectations.json

This throws the following exception (permissions):

mock-server_1  | 2022-03-18 14:15:13 5.13.0 SEVERE exception creating persisted expectations file /config/modified expectations.json 
mock-server_1  | java.nio.file.AccessDeniedException: /config/modified expectations.json
mock-server_1  |        at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
mock-server_1  |        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
mock-server_1  |        at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
mock-server_1  |        at java.base/sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219)
mock-server_1  |        at java.base/java.nio.file.Files.newByteChannel(Files.java:371)
mock-server_1  |        at java.base/java.nio.file.Files.createFile(Files.java:648)
mock-server_1  |        at org.mockserver.persistence.ExpectationFileSystemPersistence.<init>(ExpectationFileSystemPersistence.java:49)
mock-server_1  |        at org.mockserver.mock.HttpState.<init>(HttpState.java:122)
mock-server_1  |        at org.mockserver.lifecycle.LifeCycle.<init>(LifeCycle.java:55)
mock-server_1  |        at org.mockserver.netty.MockServer.<init>(MockServer.java:77)
mock-server_1  |        at org.mockserver.netty.MockServer.<init>(MockServer.java:48)
mock-server_1  |        at org.mockserver.cli.Main.main(Main.java:145)

I have also created the specified file in the folder with touch but the problem persists.

I think the problem here is the file name, which contains a space (/config/modified expectations.json). Change it to /config/modified_expectations.json.

mock-server / mockserver

When watchInitializationJson is enabled, updating expectations through expectation ID will clear all expectations #1055