muter-mutation-testing / muter

🔎 Automated mutation testing for Swift 🕳️
MIT License
491 stars 39 forks source link

Cannot write to file when run from Xcode 15 #248

Closed steveriggins closed 4 months ago

steveriggins commented 9 months ago

"Showing All Messages muterCore/fileOperations.swift:28: Fatal error: 'try!' expression unexpectedly raised an error: Error Domain=NSCocoaErrorDomain Code=513 "You don’t have permission to save the file “Sep 30, 2023 at 3/08 PM” in the folder “muter_logs”." UserInfo={NSFilePath=/Users/MYUSER/source/Muter Sandbox/muter_logs/Sep 30, 2023 at 3:08 PM, NSUnderlyingError=0x600002810030 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}}"

I compiled muter from main, started a new project with the Xcode 15 Swift Data template, added an aggregate target, and added a "muter --format xcode" run phase script.

When run from Xcode, I get the above error. When run from the command line, everything works but it takes over 30 minutes to mutate the same app.

ZevEisenberg commented 9 months ago

The 30 minute things makes sense - it has to run your test suite a bunch of times. As for permissions, I think Xcode 15 may have changed the sandbox setup but I forget how.

steveriggins commented 9 months ago

Thanks! I am just checking out techologies that might help our workflow. I work on a very large app with 1+ million lines of code and around 20 sub-projects (which causes some issues with the clone process, it only seems to clone the top level project) but even that takes awhile as it's multiple gits of git repo etc. The concept is very cool, I really like it as code coverage means almost nothing. I am refactoring some critical portions of our code so I am going to see if I can figure out how to muter test just this code and any code it might use, but the dependencies may prove a challenge.

Anything about the sandbox I can try on this end to help you out?

rakaramos commented 8 months ago

@steveriggins do you have a sample project with the issue so we can reproduce it? The xcode report format was really meant for exploration on either very small ios project or, and more ideally, macOS apps.

With mutation schemata we had a huge improvement in terms of performance (mainly the overall time Muter takes to run), but for iOS project the issue is the simulator (you can maybe try parallel testing and whatnot, but the real issue, as far as my investigations is the simulator).

My suggestion to you would be to use Muter in a UI agnostic module/project, by doing that you can run using macOS as the target and have Muter running super fast.

rakaramos commented 4 months ago

@steveriggins i'm closing this one due to inactivity, please try the last version or the version from master. feel free to reopen if the issue keeps happening. thanks!