服务器之家：专注于服务器技术及软件下载分享
分类导航

PHP教程|ASP.NET教程|Java教程|ASP教程|编程技术|正则表达式|C/C++|IOS|C#|Swift|Android|VB|R语言|JavaScript|易语言|vb.net|

服务器之家 - 编程语言 - Java教程 - 关于SpringBoot大文件RestTemplate下载解决方案

关于SpringBoot大文件RestTemplate下载解决方案

2022-02-27 15:11人无名，则可专心练剑 Java教程

这篇文章主要介绍了SpringBoot大文件RestTemplate下载解决方案,最近结合网上案例及自己总结，写了一个分片下载tuling/fileServer项目，需要的朋友可以参考下

近期基于项目上使用到的RestTemplate下载文件流，遇到1G以上的大文件，下载需要3-4分钟，因为调用API接口没有做分片与多线程， 文件流全部采用同步方式加载，性能很慢。最近结合网上案例及自己总结，写了一个分片下载tuling/fileServer项目： 1.包含同步下载文件流在浏览器加载输出相关代码； 2.包含分片多线程下载分片文件及合并文件相关代码；

另外在DownloadThread项目中使用代码完成了一个远程RestUrl请求去获取一个远端资源大文件进行多线程分片下载 到本地的一个案例，可以下载一些诸如.mp4/.avi等视频类大文件。相关代码也一并打包上传。

同步下载，支持分片下载Range主要代码：

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
@Controller
public class DownLoadController {
    private static final String UTF8 = "UTF-8";
    @RequestMapping("/download")
    public void downLoadFile(HttpServletRequest request, HttpServletResponse response) throws IOException {
        File file = new File("D:\\DevTools\\ideaIU-2021.1.3.exe");
        response.setCharacterEncoding(UTF8);
        InputStream is = null;
        OutputStream os = null;
        try {
            // 分片下载 Range表示方式 bytes=100-1000  100-
            long fSize = file.length();
            response.setContentType("application/x-download");
            String fileName = URLEncoder.encode(file.getName(), UTF8);
            response.addHeader("Content-Disposition", "attachment;filename=" + fileName);
            // 支持分片下载
            response.setHeader("Accept-Range", "bytes");
            response.setHeader("fSize", String.valueOf(fSize));
            response.setHeader("fName", fileName);
 
            long pos = 0, last = fSize - 1, sum = 0;
            if (null != request.getHeader("Range")) {
                response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT);
                String numberRange = request.getHeader("Range").replaceAll("bytes=", "");
                String[] strRange = numberRange.split("-");
                if (strRange.length == 2) {
                    pos = Long.parseLong(strRange[0].trim());
                    last = Long.parseLong(strRange[1].trim());
                    if (last > fSize-1) {
                        last = fSize - 1;
                    }
                } else {
                    pos = Long.parseLong(numberRange.replaceAll("-", "").trim());
                }
            }
            long rangeLength = last - pos + 1;
            String contentRange = new StringBuffer("bytes").append(pos).append("-").append(last).append("/").append(fSize).toString();
            response.setHeader("Content-Range", contentRange);
            response.setHeader("Content-Length", String.valueOf(rangeLength));
 
            os = new BufferedOutputStream(response.getOutputStream());
            is = new BufferedInputStream(new FileInputStream(file));
            is.skip(pos);
            byte[] buffer = new byte[1024];
            int length = 0;
            while (sum < rangeLength) {
                int readLength = (int) (rangeLength - sum);
                length = is.read(buffer, 0, (rangeLength - sum) <= buffer.length ? readLength : buffer.length);
                sum += length;
                os.write(buffer,0, length);
            }
            System.out.println("下载完成");
        }finally {
            if (is != null){
                is.close();
            }
            if (os != null){
                os.close();
            }
        }
    }
}

多线程分片下载分片文件，下载完成之后合并分片主要代码：

?
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
@RestController
public class DownloadClient {
    private static final Logger LOGGER = LoggerFactory.getLogger(DownloadClient.class);
    private final static long PER_PAGE = 1024L * 1024L * 50L;
    private final static String DOWN_PATH = "F:\\fileItem";
    ExecutorService taskExecutor = Executors.newFixedThreadPool(10);
 
    @RequestMapping("/downloadFile")
    public String downloadFile() {
        // 探测下载
        FileInfo fileInfo = download(0, 10, -1, null);
        if (fileInfo != null) {
            long pages =  fileInfo.fSize / PER_PAGE;
            for (long i = 0; i <= pages; i++) {
                Future<FileInfo> future = taskExecutor.submit(new DownloadThread(i * PER_PAGE, (i + 1) * PER_PAGE - 1, i, fileInfo.fName));
                if (!future.isCancelled()) {
                    try {
                        fileInfo = future.get();
                    } catch (InterruptedException | ExecutionException e) {
                        e.printStackTrace();
                    }
                }
            }
            return System.getProperty("user.home") + "\\Downloads\\" + fileInfo.fName;
        }
        return null;
    }
 
    class FileInfo {
        long fSize;
        String fName;
 
        public FileInfo(long fSize, String fName) {
            this.fSize = fSize;
            this.fName = fName;
        }
    }
 
    /**
     * 根据开始位置/结束位置
     * 分片下载文件，临时存储文件分片
     * 文件大小=结束位置-开始位置
     *
     * @return
     */
    private FileInfo download(long start, long end, long page, String fName) {
        File dir = new File(DOWN_PATH);
        if (!dir.exists()) {
            dir.mkdirs();
        }
        // 断点下载
        File file = new File(DOWN_PATH, page + "-" + fName);
        if (file.exists() && page != -1 && file.length() == PER_PAGE) {
            return null;
        }
        try {
            HttpClient client = HttpClients.createDefault();
            HttpGet httpGet = new HttpGet("http://127.0.0.1:8080/download");
            httpGet.setHeader("Range", "bytes=" + start + "-" + end);
            HttpResponse response = client.execute(httpGet);
            String fSize = response.getFirstHeader("fSize").getValue();
            fName = URLDecoder.decode(response.getFirstHeader("fName").getValue(), "UTF-8");
            HttpEntity entity = response.getEntity();
            InputStream is = entity.getContent();
            FileOutputStream fos = new FileOutputStream(file);
            byte[] buffer = new byte[1024];
            int ch;
            while ((ch = is.read(buffer)) != -1) {
                fos.write(buffer, 0, ch);
            }
            is.close();
            fos.flush();
            fos.close();
            // 最后一个分片
            if (end - Long.parseLong(fSize) > 0) {
                // 开始合并文件
                mergeFile(fName, page);
            }
 
            return new FileInfo(Long.parseLong(fSize), fName);
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }
 
    private void mergeFile(String fName, long page) {
        File file = new File(DOWN_PATH, fName);
        try {
            BufferedOutputStream os = new BufferedOutputStream(new FileOutputStream(file));
            for (long i = 0; i <= page; i++) {
                File tempFile = new File(DOWN_PATH, i + "-" + fName);
                while (!file.exists() || (i != page && tempFile.length() < PER_PAGE)) {
                    try {
                        Thread.sleep(100);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                }
                byte[] bytes = FileUtils.readFileToByteArray(tempFile);
                os.write(bytes);
                os.flush();
                tempFile.delete();
            }
            File testFile = new File(DOWN_PATH, -1 + "-null");
            testFile.delete();
            os.flush();
            os.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    /**
     * 获取远程文件尺寸
     */
    private long getRemoteFileSize(String remoteFileUrl) throws IOException {
        long fileSize = 0;
        HttpURLConnection httpConnection = (HttpURLConnection) new URL(remoteFileUrl).openConnection();
        //使用HEAD方法
        httpConnection.setRequestMethod("HEAD");
        int responseCode = httpConnection.getResponseCode();
        if (responseCode >= 400) {
            LOGGER.debug("Web服务器响应错误!");
            return 0;
        }
        String sHeader;
        for (int i = 1;; i++) {
            sHeader = httpConnection.getHeaderFieldKey(i);
            if (sHeader != null && sHeader.equals("Content-Length")) {
                LOGGER.debug("文件大小ContentLength:" + httpConnection.getContentLength());
                fileSize = Long.parseLong(httpConnection.getHeaderField(sHeader));
                break;
            }
        }
        return fileSize;
    }
 
    class DownloadThread implements Callable<FileInfo> {
        long start;
        long end;
        long page;
        String fName;
 
        public DownloadThread(long start, long end, long page, String fName) {
            this.start = start;
            this.end = end;
            this.page = page;
            this.fName = fName;
        }
 
        @Override
        public FileInfo call() {
            return download(start, end, page, fName);
        }
    }
}

代码都在本地亲测（已修复Bug）可用，目前比较欠缺的是没有实现在分片下载时对应浏览器进行下载展示，需要暂存在本地磁盘目录。 目前将代码开源，希望能有更好解决方案的Coder Fork支持！也欢迎Star捧场。

博文参考了图灵学院相关的分片下载案例教程，并修改了部分代码实现：

WebUploader--基于SpringBoot搭建，Java文件上传下载高阶实战

本文代码已上传至GitHub：

BurstDownload

到此这篇关于SpringBoot大文件RestTemplate下载解决方案的文章就介绍到这了,更多相关SpringBoot RestTemplate下载内容请搜索服务器之家以前的文章或继续浏览下面的相关文章希望大家以后多多支持服务器之家！

原文链接：https://www.cnblogs.com/yif0118/p/15456666.html

  • springboot
  • 下载
  • RestTemplate

    • 延伸 · 阅读

    精彩推荐
    最近更新
    编辑推荐
    阅读排行
    热门标签
    501 　 406 　 415 　 1433 　 2000 　 401 　 330 　 GOF23 　 堆栈类 　 微信公众号 　 注册服务 　 嵌套循环 　 critical 　 consul 　 spring 　 集合 　 volatile 　 Hibernate 　 Java 　 new 　 double 　 泛型 　 Java编程 　 遍历 　 设计模式 　 supplyAsync 　 制表符 　 回车 　 写入文件 　 JAVA8 　
    © 2019-2024 服务器之家(www.zzvips.com) 版权所有关于我们联系我们版权申明网站地图