alphadose / haxmap

Fastest and most memory efficient golang concurrent hashmap
MIT License
892 stars 45 forks source link

Save to file, load from file #45

Closed Noname400 closed 5 months ago

Noname400 commented 6 months ago

Pls, help

semihbkgr commented 6 months ago

Hi @Noname400

You can easily utilize JSON serialization to write the map to a file and read it back from the file. The map type already implements encoding/json.Marshaler and Unmarshaler.

Here is an example:

func main() {
    mep := haxmap.New[int, string]()
    mep.Set(1, "one")
    mep.Set(2, "two")
    mep.Set(3, "three")

    b, err := json.Marshal(mep)
    if err != nil {
        panic(err)
    }
    path := filepath.Join(os.TempDir(), "haxmap")
    err = os.WriteFile(path, b, 0644)
    if err != nil {
        panic(err)
    }

    b, err = os.ReadFile(path)
    if err != nil {
        panic(err)
    }
    hmap := haxmap.New[int, string]()
    err = json.Unmarshal(b, &hmap)
    if err != nil {
        panic(err)
    }
    hmap.ForEach(func(k int, v string) bool {
        fmt.Println(k, v)
        return true
    })
}
Noname400 commented 6 months ago

Is it possible to write and read in binary format? The speed of reading from a file is very important to me. Search speed is very important to me.

alphadose commented 6 months ago

Is it possible to write and read in binary format?

to achieve this you would need to implement your custom binary marshal/unmarshal functions to haxmap

semihbkgr commented 6 months ago

@Noname400 What do you mean by search? If you are referring to searching for data in a file, you probably need a more specific disk-based data structure. Alternatively, you can implement a custom serializer to meet your specific needs.

Noname400 commented 6 months ago

now I'm using a bluemilter, but it gives false positive results. I'm looking for a structure to search in 90 million records. the search will be in memory and not in a file