alpstable / gidari

Transport web data to local/remote storage using Gidari
Apache License 2.0
29 stars 29 forks source link

Handle duplicate connection strings in configuration file #163

Closed prestonvasquez closed 1 year ago

prestonvasquez commented 2 years ago

A user could create a configuration file with duplicate connection strings like this:

url: https://official-joke-api.appspot.com
connectionStrings: 
  - mongodb://mongo1:27017/jokes
  - mongodb://mongo1:27017/jokes
rateLimit:
  burst: 5
  period: 1
requests:
  - endpoint: /random_joke

If this happens, we need to log a warning in transport.Config#repos that the user did this, and only create a repo for one of the connection strings.

krishnaindani commented 2 years ago

hey, could I take a stab at this? Would like to work if open.

prestonvasquez commented 2 years ago

@krishnaindani You're assigned!

krishnaindani commented 2 years ago

Thanks

krishnaindani commented 2 years ago

The way I was thinking is to use a map[string]bool while we loop to make newTX here https://github.com/alpine-hodler/gidari/blob/main/internal/transport/transport.go#L269 and maintaining a dns entry in map that had we have already looped over. And if duplicated just log it. How does this sounds or would you be able to recommend any better idea here?

prestonvasquez commented 2 years ago

@krishnaindani That sounds reasonable to me! In order to maintain the complexity requirements for Hacktoberfest, do you think you could take a look at Issue #180 and combine the two into one PR?

krishnaindani commented 2 years ago

Sure will take a look

krishnaindani commented 2 years ago

Could you assign me that issue?

prestonvasquez commented 2 years ago

@krishnaindani Thank you! Since you have not made your first contribution can you comment on that issue so that you are in the scope of assignable engineers?

prestonvasquez commented 2 years ago

@krishnaindani Are you still interested in pursuing this issue?

prestonvasquez commented 1 year ago

Closed by #360