ikeikeikeike / go-sitemap-generator

go-sitemap-generator is the easiest way to generate Sitemaps in Go
https://github.com/ikeikeikeike/go-sitemap-generator
MIT License
218 stars 65 forks source link

Change map into 2d interface slices so the generated xml will be in order #24

Closed kosto1992 closed 5 years ago

kosto1992 commented 5 years ago

We needed to change the map into slices in go-sitemap-generator library and to disable concurrency. The change was needed because map is storing values in random order, so the output XML is not valid against XSDs. This change was done for the purposes of The Economist.

The library now allows to set the max number of processors (concurrency). If it is set to 1 it will not use concurrency, if it is set to lower then 1, it will use maxCPUs available.

I used 2d interfaces slice [][]interface{} - it will enable to process sitemap same way as they were processed. The format how to write the item when adding into sitemap has changed due to this change.

The change to slices is faster forPerformance 500 in Benchmark test.

Origin benchmark (with usage of map):

goarch: amd64
pkg: github.com/EconomistDigitalSolutions/go-sitemap-generator/stm
BenchmarkGenerateXML-4             1    16766094832 ns/op   8880786240 B/op 117012061 allocs/op
PASS
ok      github.com/EconomistDigitalSolutions/go-sitemap-generator/stm   16.786s
Success: Benchmarks passed.

New benchmark (with usage of 2d slice of interfaces):


goarch: amd64
pkg: github.com/EconomistDigitalSolutions/cp-worker-format-sitemaps/vendor/github.com/kosto1992/go-sitemap-generator/stm
BenchmarkGenerateXML-4             1    15243760666 ns/op   8550720128 B/op 122268424 allocs/op
PASS
ok      github.com/EconomistDigitalSolutions/cp-worker-format-sitemaps/vendor/github.com/kosto1992/go-sitemap-generator/stm 15.277s
Success: Benchmarks passed.```
ikeikeikeike commented 5 years ago

thx, I'll make sure this one.

buddhamagnet commented 5 years ago

@ikeikeikeike this is insanely useful for me, any ETA on a merge as we're vendoring the fork right now.

ikeikeikeike commented 5 years ago

The new location https://github.com/ikeikeikeike/go-sitemap-generator/tree/v2 A v2 branch will be set default branch instead of a master branch later.

Many thanks.