dromara / x-file-storage

一行代码将文件存储到 本地、FTP、SFTP、WebDAV、谷歌云存储、阿里云OSS、华为云OBS、七牛云Kodo、腾讯云COS、百度云 BOS、又拍云USS、MinIO、 AWS S3、FastDFS、 Azure Blob Storage、金山云 KS3、美团云 MSS、京东云 OSS、天翼云 OOS、移动云 EOS、沃云 OSS、 网易数帆 NOS、Ucloud US3、青云 QingStor、平安云 OBS、首云 OSS、IBM COS、其它兼容 S3 协议的平台。后续即将支持 Samba、NFS
https://x-file-storage.xuyanwu.cn/
Apache License 2.0
1.68k stars 260 forks source link

上传的文件 MD5 都一样 #269

Closed ion1ze closed 2 months ago

ion1ze commented 2 months ago

调用方式如下:

@Operation(summary = "上传文件", description = "上传文件")
@PostMapping(value = "/file/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public Result<InfraFileUploadResponse> upload(@Parameter(description = "文件", required = true)
                                              @NotNull(message = "上传文件不能为空")
                                              @RequestParam MultipartFile file) {
    String date = DateUtil.format(LocalDateTime.now(), "yyyy/MM/dd");
    String path = String.format(PATH_FORMAT, date);
    FileInfo fileInfo = this.fileStorageService.of(file)
            .setHashCalculatorMd5()
            .setPath(path)
            .upload();

    InfraFileUploadResponse data = InfraFileUploadResponse.from(fileInfo);
    log.info("MD5:{}", fileInfo.getHashInfo().getMd5());
    return Result.success("上传文件成功", data);
}

上传了几个不同的文件 Snipaste_2024-07-05_14-48-04

1171736840 commented 2 months ago

用的哪个版本

ion1ze commented 2 months ago

用的哪个版本 jdk:17 springboot:3.2.4 x-file-storage:2.2.0

1171736840 commented 2 months ago

好的,感谢你的反馈,我这边看一下

1171736840 commented 2 months ago

我这边测试没问题,能否提供更多配置文件及复现代码?

ion1ze commented 2 months ago

配置文件:

dromara:
  x-file-storage:
    default-platform: local-plus
    local-plus:
      - platform: local-plus
        enable-storage: true
        enable-access: true
        domain: http://localhost:8080/uploads/
        path-patterns: /uploads/**
        storage-path: ${user.dir}/.data/uploads/

FileRecord 实现类:

@Slf4j
@Service
@RequiredArgsConstructor
public class InfraFileServiceImpl extends ServiceImpl<InfraFileMapper, InfraFile> implements InfraFileService {

    public InfraFile toInfraFile(FileInfo source) throws JsonProcessingException {
        InfraFile target = new InfraFile();
        target.setUrl(source.getUrl());
        target.setSize(source.getSize());
        target.setFilename(source.getFilename());
        target.setType(source.getContentType());
        target.setOriginalFilename(source.getOriginalFilename());
        target.setBasePath(source.getBasePath());
        target.setPath(source.getPath());
        target.setExtension(source.getExt());
        target.setPlatform(source.getPlatform());
        target.setChecksum(JsonUtil.toJsonString(source.getHashInfo()));
        target.setObjectId(source.getObjectId());
        target.setObjectType(source.getObjectType());
        target.setMetadata(JsonUtil.toJsonString(source.getMetadata()));
        target.setUserMetadata(JsonUtil.toJsonString(source.getUserMetadata()));
        target.setAcl(JsonUtil.toJsonString(source.getFileAcl()));
        target.setAttributes(JsonUtil.toJsonString(source.getAttr()));
        // 缩略图
        target.setThumbnailUrl(source.getThUrl());
        target.setThumbnailSize(source.getThSize());
        target.setThumbnailFilename(source.getFilename());
        target.setThumbnailType(source.getThContentType());
        target.setThumbnailMetadata(JsonUtil.toJsonString(source.getThMetadata()));
        target.setThumbnailUserMetadata(JsonUtil.toJsonString(source.getThUserMetadata()));
        target.setThumbnailAcl(JsonUtil.toJsonString(source.getThFileAcl()));
        // 分片
        target.setUploadId(source.getUploadId());
        target.setUploadStatus(source.getUploadStatus());
        return target;
    }

    public FileInfo toFileInfo(InfraFile source) throws JsonProcessingException {
        FileInfo target = new FileInfo();
        target.setUrl(source.getUrl());
        target.setSize(source.getSize());
        target.setFilename(source.getFilename());
        target.setContentType(source.getType());
        target.setOriginalFilename(source.getOriginalFilename());
        target.setBasePath(source.getBasePath());
        target.setPath(source.getPath());
        target.setExt(source.getExtension());
        target.setPlatform(source.getPlatform());
        target.setHashInfo(JsonUtil.parseObject(source.getChecksum(), HashInfo.class));
        target.setObjectId(source.getObjectId());
        target.setObjectType(source.getObjectType());
        target.setMetadata(JsonUtil.parseObjectMap(source.getMetadata(), String.class, String.class));
        target.setUserMetadata(JsonUtil.parseObjectMap(source.getUserMetadata(), String.class, String.class));
        target.setAttr(JsonUtil.parseObject(source.getAttributes(), Dict.class));
        // 缩略图
        target.setThUrl(source.getThumbnailUrl());
        target.setThSize(source.getThumbnailSize());
        target.setThFilename(source.getThumbnailFilename());
        target.setThContentType(source.getThumbnailType());
        target.setThMetadata(JsonUtil.parseObjectMap(source.getThumbnailMetadata(), String.class, String.class));
        target.setThUserMetadata(JsonUtil.parseObjectMap(source.getThumbnailUserMetadata(), String.class, String.class));
        // 分片
        target.setUploadId(source.getUploadId());
        target.setUploadStatus(source.getUploadStatus());
        return target;
    }

    @Override
    @SneakyThrows
    public boolean save(FileInfo fileInfo) {
        InfraFile entity = toInfraFile(fileInfo);
        boolean result = this.save(entity);
        if (result) {
            fileInfo.setId(String.valueOf(entity.getId()));
        }
        return result;
    }

    @Override
    @SneakyThrows
    public void update(FileInfo fileInfo) {
        InfraFile entity = toInfraFile(fileInfo);
        super.updateById(entity);
    }

    @Override
    public FileInfo getByUrl(String url) {
        LambdaQueryWrapper<InfraFile> queryWrapper = new LambdaQueryWrapper<>();
        queryWrapper.eq(InfraFile::getUrl, url);
        return Optional.ofNullable(super.getOne(queryWrapper))
                .map(source -> {
                    try {
                        return toFileInfo(source);
                    } catch (JsonProcessingException e) {
                        log.error("JsonProcessingException", e);
                        return null;
                    }
                }).orElse(null);
    }

    @Override
    public boolean delete(String url) {
        LambdaQueryWrapper<InfraFile> queryWrapper = new LambdaQueryWrapper<>();
        queryWrapper.eq(InfraFile::getUrl, url);
        return super.remove(queryWrapper);
    }

    @Override
    public void saveFilePart(FilePartInfo filePartInfo) {

    }

    @Override
    public void deleteFilePartByUploadId(String s) {

    }

    @Override
    public PaginationResponse<InfraFileDetailsResponse> pagination(InfraFilePaginationRequest request) {
        LambdaQueryWrapper<InfraFile> queryWrapper = new LambdaQueryWrapper<>();

        Page<InfraFile> page = super.page(Page.of(request.getPageNumber(), request.getPageSize()), queryWrapper);
        List<InfraFileDetailsResponse> records = page.getRecords().stream()
                .map(InfraFileDetailsResponse::from)
                .toList();
        return PaginationResponse.of(page.getCurrent(), page.getSize(), page.getTotal(), records);
    }
}
ion1ze commented 2 months ago

依赖版本:

<hutool.version>5.8.26</hutool.version>
<guava.version>33.2.0-jre</guava.version>
<sa-token.version>1.37.0</sa-token.version>
<transmittable-thread-local.version>2.14.5</transmittable-thread-local.version>
<springdoc.version>2.3.0</springdoc.version>
<caffeine.version>3.1.8</caffeine.version>
<x-file-storage.version>2.2.0</x-file-storage.version>
<retrofit2.version>2.9.0</retrofit2.version>
<okhttp3.version>4.9.3</okhttp3.version>
<!--<flyway.version>7.15.0</flyway.version>-->
<mybatis-plus.version>3.5.5</mybatis-plus.version>
<mybatis-plus-join.version>1.4.9</mybatis-plus-join.version>
<mysql.version>8.2.0</mysql.version>
<zxing.version>3.5.3</zxing.version>
<java.version>17</java.version>
1171736840 commented 2 months ago

加个QQ群 515706495 咱们详细讨论一下

1171736840 commented 2 months ago

原因是 MultipartFile 支持移动文件功能,在本地存储平台下会不读取输入流,就导致没有计算hash出现此问题。 目前dev分支已修复此问题并发布了快照版本 2.2.1-SNAPSHOT 可以使用,也可以使用下面的上传方式来避免此问题

FileInfo fileInfo = this.fileStorageService.of(file.getInputStream(), file.getName(), file.getContentType(), file.getSize())
                .setHashCalculatorMd5()
                .upload();