Open andk opened 2 months ago
From what I can tell this happens because we now do copy-on-write for more values.
In this case $data
at https://metacpan.org/release/PLICEASE/UUID-FFI-0.11/source/lib/UUID/FFI.pm#L118
The tests that fail here sort the uuids, then call as_hex() on each uuid, as_hex() takes a pointer to the buffer of $data
, but since that is now CoW it's the same buffer each time, so the map ends up overwriting that buffer as it works its way through the list, finally producing a list of all the same hex UUID.
I expect the real fix here is for FFI::Platypus to implement some sort of string buffer parameter type and un-CoW the buffer with SvPV_force().
Another approach that could be done by core would be to make pack "P"
SvPV_force() the buffer, making P less[1] of a dangerous mess. This would slow down pack "P" a little, or increase memory usage (depending on the size of the PV for the SV), but it would make pack "P" safer, and perhaps more like what people expect.
[1] less, but still dangerous
I am happy to put the fix in either pack "P"
or in FFI::Platypus
. There is already a FFI::Platypus::Buffer
module which does this same thing and the scalar_to_pointer
function could be pretty easily reimplemented as XS if necessary, and UUID::FFI
should tbh be using that instead of rolling its own unpack/pack. My intuition is that pack "P"
would be better being safer even if it does cost a little more.
One problem with always doing a force normal/SvPV_force() is that it throws for a readonly SV, which may break existing code that uses pack "P" without needing a writeable pointer.
Another issue is if the SV has set magic, that set magic is never called when using the mechanism that UUID::FFI uses. I don't see any change that can be done for that in perl.
Always doing a SvPV_force() breaks tests, and would break at least one example in perlpacktut.pod
I just discovered another module affected by this change: MONS/Dash-Leak-0.06.tar.gz
Sample fail report: http://www.cpantesters.org/cpan/report/1dd7ca24-3f14-11ef-ab0f-0e74c8b2e081
Also affected: TMM/HEAT-Crypto-0.07.tar.gz http://www.cpantesters.org/cpan/report/fbd71cd8-3c50-11ef-86c6-156034470c55
Also affected: TMM/HEAT-Crypto-0.07.tar.gz http://www.cpantesters.org/cpan/report/fbd71cd8-3c50-11ef-86c6-156034470c55
That just needs a single SvTHINKFIRST
to be fixed.
Also affected: NGLENN/Algorithm-AM-3.12.tar.gz http://www.cpantesters.org/cpan/report/90f70a84-3b21-11ef-897d-51c950bdeb56
@richardleach, the breakage being discussed in this ticket has been attributed to one of your commits in the last dev cycle.
commit 06e421c559c63975f29c35ba3588a0e6b0c75eca
Author: Richard Leach <rich+perl@hyphen-dash-hyphen.info>
AuthorDate: Fri Dec 9 23:42:32 2022 +0000
Commit: Richard Leach <rich+perl@hyphen-dash-hyphen.info>
CommitDate: Tue Jun 11 21:20:02 2024 +0100
S_fold_constants: remove early SvREADONLY(sv) to allow SvIsCOW(sv)
Would you care to comment? Thanks.
@richardleach, the breakage being discussed in this ticket has been attributed to one of your commits in the last dev cycle.
I've been away from my dev VM for the past ~10 days and won't be properly back at it for another 10. I can look into these at that point.
From what's been mentioned so far, it doesn't sound like a technical bug has been introduced into core, rather that the previous no-SvIsCOW behaviour has been relied upon. (Although I've not had time to sit down and get my head around the implications for pack, nor SvPV_force() breaking tests, as per @tonycoz's comments above.)
AM.xs
isn't prepared for COWed buffers, so that's where I'd go looking.Noted that there are quick 'n' dirty pure-perl ways to defeat COW in the UUID::FFI and Dash::Leak cases by lightly permuting the buffer to un-COW it. For example:
my $data = ("x" x 35)."w"; $data++;
That's obviously introducing an extra operation and obfuscating the code, so isn't ideal, but might help in some cases.
That's obviously introducing an extra operation and obfuscating the code, so isn't ideal, but might help in some cases.
Someone might come along and optimize those two steps into a constant assignment, it's not like $data
is being passed anywhere that might make it magical.
I'll work on a patch to SvPV_force() non-READONLY SVs, but that's only safer, not safe.
That will fix the mechanism in the specific use by UUID::FFI, but it's not a general fix, since modification through that pointer doesn't call set magic either, so if a tied SV is passed in the changes will evaporate on the next fetch of the SV.
It won't help any of the other failures here.
nor SvPV_force() breaking tests, as per @tonycoz's comments above.
Some tests pass in constant strings which SvPV_force() reasonably throws on, so I'll make it only force for non-READONLY.
Thanks @tonycoz. I can work on patches for the other failures next week.
I've opened https://rt.cpan.org/Ticket/Display.html?id=154991 for Dash::Leak.
Also affected: TMM/HEAT-Crypto-0.07.tar.gz http://www.cpantesters.org/cpan/report/fbd71cd8-3c50-11ef-86c6-156034470c55
That just needs a single
SvTHINKFIRST
to be fixed.
I tried modifying *S_get_key_buffer()
to do SV_CHECK_THINKFIRST(var);
immediately prior to the BYTE* buff = (BYTE*) SvPV(var, len);
line.
This looks to fix the 'decrypt shared'
test failure. It does not fix the 'verify'
failure. Will dig further.
Description
with v5.41.0-13-g06e421c559 a test started to fail for PLICEASE/UUID-FFI-0.11.tar.gz
Sample fail report: http://www.cpantesters.org/cpan/report/10ab0748-3a79-11ef-9583-ab3925c06771
Bisect says:
@plicease , you may want to watch this issue.
Steps to Reproduce
cpan -i PLICEASE/UUID-FFI-0.11.tar.gz
Expected behavior
Should compile and test OK
Perl configuration