ariovistus / pyd

Interoperability between Python and D
MIT License
158 stars 32 forks source link

Crash in MultiIndexContainer after change in druntime getHash for pointers #104

Closed atilaneves closed 5 years ago

atilaneves commented 5 years ago

Going from dmd 2.081.2 to dmd 2.082.0 makes the code at the bottom crash. The reason is that TypeInfo_Pointer.getHash was changed in druntime, and that's used in the MultiIndexContainer for DStruct_Py_Mapping in the declaration

alias MultiIndexContainer!(Mapping, IndexedBy!(
                HashedUnique!("a.d"), "d",
                HashedUnique!("a.py"), "python"),
            MallocAllocator, MutableView)
        Container;

HashedUnique above is:

template HashedUnique(alias KeyFromValue="a",
        alias Hash="typeid(a).getHash(&a)", alias Eq="a==b"){
    alias Hashed!(false, KeyFromValue, Hash, Eq) HashedUnique;
}

It's the typeid(a).getHash(&a) that has changed implementation, and that caused pyd to crash with a failing assertion on multi_index.d:230 - the next pointer has an integer value.

It could be argued that this is a bug in druntime instead - but given how small the change was and that the new hash function could return 42 for every pointer and technically work I don't think so. Especially given the complexity of the implementation of MultiIndexContainer.

Code to reproduce:

// dub.sdl
name "pyd_druntime"
targetType "dynamicLibrary"
dependency "autowrap:python" version="~>0.1.1"
postBuildCommands "mv libpyd_druntime.so pyd_druntime.so" platform="posix"

configuration "default" {
    subConfiguration "autowrap:python" "python36"
}
//app.d
import autowrap.python;
mixin(
    wrapAll(
        LibraryName("pyd_druntime"),
        Modules("api"),
    )
);
// api.d
struct PriceBar {
    import std.datetime: DateTime;

    DateTime date;
    double open, high, low, close;

    this(DateTime date, double open, double high, double low, double close) {
        this.date = date;
        this.open = open;
        this.high = high;
        this.low = low;
        this.close = close;
    }
}
# test_pyd_druntime.py
from pyd_druntime import PriceBar
from datetime import datetime as Date

def test_oops():
    bars = the_price_bars()

def the_price_bars():
    return [
        PriceBar(Date(2004, 8, 19), 100.01, 104.06, 95.96, 100.335),
        PriceBar(Date(2004, 8, 20), 101.01, 109.08, 100.5, 108.31),
        PriceBar(Date(2004, 8, 23), 110.76, 113.48, 109.05, 109.4),
        PriceBar(Date(2004, 8, 24), 111.24, 111.6, 103.57, 104.87),
        PriceBar(Date(2004, 8, 25), 104.76, 108.0, 103.88, 106.0),
        PriceBar(Date(2004, 8, 26), 104.95, 107.95, 104.66, 107.91),
        PriceBar(Date(2004, 8, 27), 108.1, 108.62, 105.69, 106.15),
        PriceBar(Date(2004, 8, 30), 105.28, 105.49, 102.01, 102.01),
        PriceBar(Date(2004, 8, 31), 102.32, 103.71, 102.16, 102.37),
        PriceBar(Date(2004, 9, 1), 102.7, 102.97, 99.67, 100.25),
        PriceBar(Date(2004, 9, 2), 99.09, 102.37, 98.94, 101.51),
        PriceBar(Date(2004, 9, 3), 100.95, 101.74, 99.32, 100.01),
        PriceBar(Date(2004, 9, 7), 101.01, 102.0, 99.61, 101.58),
        PriceBar(Date(2004, 9, 8), 100.74, 103.03, 100.5, 102.3),
        PriceBar(Date(2004, 9, 9), 102.5, 102.71, 101.0, 102.31),
        PriceBar(Date(2004, 9, 10), 101.47, 106.56, 101.3, 105.33),
        PriceBar(Date(2004, 9, 13), 106.63, 108.41, 106.46, 107.5),
        PriceBar(Date(2004, 9, 14), 107.44, 112.0, 106.79, 111.49),
        PriceBar(Date(2004, 9, 15), 110.56, 114.23, 110.2, 112.0),
        PriceBar(Date(2004, 9, 16), 112.34, 115.8, 111.65, 113.97),
        PriceBar(Date(2004, 9, 17), 114.42, 117.49, 113.55, 117.49),
        PriceBar(Date(2004, 9, 20), 116.95, 121.6, 116.77, 119.36),
        PriceBar(Date(2004, 9, 21), 120.2, 120.42, 117.51, 117.84),
        PriceBar(Date(2004, 9, 22), 117.45, 119.67, 116.81, 118.38),
        PriceBar(Date(2004, 9, 23), 118.84, 122.63, 117.02, 120.82),
        PriceBar(Date(2004, 9, 24), 120.97, 124.1, 119.76, 119.83),
        PriceBar(Date(2004, 9, 27), 119.56, 120.88, 117.8, 118.26),
        PriceBar(Date(2004, 9, 28), 121.15, 127.4, 120.21, 126.86),
        PriceBar(Date(2004, 9, 29), 126.53, 135.02, 126.23, 131.08),
        PriceBar(Date(2004, 9, 30), 129.899, 132.3, 129.0, 129.6),
        PriceBar(Date(2004, 10, 1), 130.8, 134.24, 128.9, 132.58),
        PriceBar(Date(2004, 10, 4), 135.275, 136.87, 134.03, 135.06),
        PriceBar(Date(2004, 10, 5), 134.66, 138.53, 132.24, 138.37),
        PriceBar(Date(2004, 10, 6), 137.67, 138.45, 136.0, 137.08),
        PriceBar(Date(2004, 10, 7), 136.56, 139.88, 136.55, 138.85),
        PriceBar(Date(2004, 10, 8), 138.73, 139.68, 137.02, 137.73),
        PriceBar(Date(2004, 10, 11), 137.01, 138.86, 133.85, 135.26),
        PriceBar(Date(2004, 10, 12), 134.49, 137.61, 133.4, 137.4),
        PriceBar(Date(2004, 10, 13), 143.24, 143.55, 140.08, 140.9),
        PriceBar(Date(2004, 10, 14), 141.02, 142.38, 138.56, 142.0),
        PriceBar(Date(2004, 10, 15), 144.95, 145.5, 141.95, 144.11),
        PriceBar(Date(2004, 10, 18), 143.12, 149.2, 141.21, 149.16),
        PriceBar(Date(2004, 10, 19), 150.5, 152.4, 147.35, 147.94),
        PriceBar(Date(2004, 10, 20), 147.94, 148.99, 139.6, 140.49),
        PriceBar(Date(2004, 10, 21), 144.13, 150.13, 141.62, 149.38),
        PriceBar(Date(2004, 10, 22), 170.435, 180.17, 164.08, 172.43),
        PriceBar(Date(2004, 10, 25), 176.28, 194.43, 172.55, 187.4),
        PriceBar(Date(2004, 10, 26), 186.449, 192.64, 180.0, 181.8),
        PriceBar(Date(2004, 10, 27), 182.509, 189.52, 181.77, 185.97),
        PriceBar(Date(2004, 10, 28), 186.63, 194.39, 185.6, 193.3),
        PriceBar(Date(2004, 10, 29), 198.87, 199.95, 190.6, 190.64),
        PriceBar(Date(2004, 11, 1), 193.5, 197.67, 191.27, 196.03),
        PriceBar(Date(2004, 11, 2), 198.73, 199.25, 193.34, 194.87),
        PriceBar(Date(2004, 11, 3), 198.19, 201.6, 190.75, 191.67),
        PriceBar(Date(2004, 11, 4), 188.25, 190.4, 183.35, 184.7),
        PriceBar(Date(2004, 11, 5), 182.0, 182.3, 168.55, 169.35),
        PriceBar(Date(2004, 11, 8), 170.86, 175.44, 169.4, 172.55),
        PriceBar(Date(2004, 11, 9), 174.1, 175.2, 165.27, 168.7),
        PriceBar(Date(2004, 11, 10), 170.67, 172.52, 166.33, 167.86),
        PriceBar(Date(2004, 11, 11), 169.32, 183.75, 167.57, 183.02),
        PriceBar(Date(2004, 11, 12), 185.23, 189.8, 177.4, 182.0),
        PriceBar(Date(2004, 11, 15), 180.45, 188.32, 178.75, 184.87),
        PriceBar(Date(2004, 11, 16), 174.39, 179.47, 170.83, 172.545),
        PriceBar(Date(2004, 11, 17), 169.02, 177.5, 169.0, 172.5),
        PriceBar(Date(2004, 11, 18), 170.4, 174.42, 165.73, 167.54),
        PriceBar(Date(2004, 11, 19), 169.06, 169.98, 166.52, 169.4),
        PriceBar(Date(2004, 11, 22), 168.6, 169.5, 161.31, 165.1),
        PriceBar(Date(2004, 11, 23), 167.915, 170.83, 166.5, 167.52),
        PriceBar(Date(2004, 11, 24), 174.84, 177.21, 172.51, 174.76),
        PriceBar(Date(2004, 11, 26), 175.8, 180.03, 175.32, 179.39),
        PriceBar(Date(2004, 11, 29), 180.255, 182.95, 177.51, 181.05),
        PriceBar(Date(2004, 11, 30), 180.7, 183.0, 180.25, 181.98),
        PriceBar(Date(2004, 12, 1), 181.77, 182.5, 179.55, 179.96),
        PriceBar(Date(2004, 12, 2), 179.9, 181.51, 178.55, 179.4),
        PriceBar(Date(2004, 12, 3), 180.0, 181.06, 177.6, 180.4),
        PriceBar(Date(2004, 12, 6), 179.13, 180.7, 176.02, 176.29),
        PriceBar(Date(2004, 12, 7), 176.0, 176.2, 170.55, 171.43),
        PriceBar(Date(2004, 12, 8), 170.57, 173.68, 168.73, 169.98),
        PriceBar(Date(2004, 12, 9), 170.25, 173.5, 168.47, 173.43),
        PriceBar(Date(2004, 12, 10), 173.43, 174.88, 171.29, 171.65),
        PriceBar(Date(2004, 12, 13), 172.03, 173.18, 169.45, 170.45),
        PriceBar(Date(2004, 12, 14), 171.0, 178.82, 169.6, 178.69),
        PriceBar(Date(2004, 12, 15), 177.99, 180.69, 176.66, 179.78),
        PriceBar(Date(2004, 12, 16), 177.22, 180.49, 175.95, 176.47),
        PriceBar(Date(2004, 12, 17), 176.76, 180.5, 176.55, 180.08),
        PriceBar(Date(2004, 12, 20), 182.0, 188.46, 181.87, 185.02),
        PriceBar(Date(2004, 12, 21), 186.28, 187.88, 183.4, 183.75),
        PriceBar(Date(2004, 12, 22), 183.9, 186.85, 183.01, 186.3),
        PriceBar(Date(2004, 12, 23), 187.45, 188.6, 186.0, 187.9),
        PriceBar(Date(2004, 12, 27), 189.43, 193.3, 189.1, 191.91),
        PriceBar(Date(2004, 12, 28), 192.11, 193.55, 191.01, 192.76),
        PriceBar(Date(2004, 12, 29), 191.97, 193.52, 191.78, 192.9),
        PriceBar(Date(2004, 12, 30), 192.97, 198.23, 191.85, 197.6),
        PriceBar(Date(2004, 12, 31), 199.23, 199.88, 192.56, 192.79),
        PriceBar(Date(2005, 1, 3), 197.4, 203.64, 195.46, 202.71),
        PriceBar(Date(2005, 1, 4), 201.33, 202.93, 193.48, 194.5),
        PriceBar(Date(2005, 1, 5), 193.45, 196.9, 192.23, 193.51),
        PriceBar(Date(2005, 1, 6), 195.25, 195.9, 187.72, 188.55),
        PriceBar(Date(2005, 1, 7), 190.64, 194.25, 188.78, 193.85),
        PriceBar(Date(2005, 1, 10), 194.5, 198.1, 191.83, 195.06),
        PriceBar(Date(2005, 1, 11), 195.62, 197.71, 193.18, 193.54)
    ]