Enable Deduplicated Pin Size Calculation

[bonedaddy]

For pinning new items, it might be worth investigating how to calculate a deduplicated cost of the references. While storing the data would get deduplicated storage costs on disk, there is a chance that in some cases we'll charge extra:

Example of how to do it:

// DedupAndCalculatePinSize is used to remove duplicate refers to objects for a more accurate pin size cost
func (im *IpfsManager) DedupAndCalculatePinSize(hash string) (int64, error) {
    // format a multiaddr api to connect to
    parsedIP := strings.Split(im.nodeAPIAddr, ":")
    multiAddrIP := fmt.Sprintf("/ip4/%s/tcp/%s", parsedIP[0], parsedIP[1])
    outBytes, err := exec.Command("ipfs", fmt.Sprintf("--api=%s", multiAddrIP), "refs", "--recursive", "--unique", hash).Output()
    if err != nil {
        return 0, err
    }
    scanner := bufio.NewScanner(strings.NewReader(string(outBytes)))
    var refsArray []string
    for scanner.Scan() {
        refsArray = append(refsArray, scanner.Text())
    }
    var calculatedRefSize int
    for _, ref := range refsArray {
        refStats, err := im.Stat(ref)
        if err != nil {
            return 0, err
        }
        calculatedRefSize = calculatedRefSize + refStats.DataSize
    }
    return int64(calculatedRefSize), nil
}