Tomcat 系列篇十二-番外介绍下 Tomcat 的上传文件限制

发表于 分类于 ， ， 阅读次数： 阅读次数： Disqus：

最近同学在把 springboot 升级到 2.x 版本的过程中碰到了小问题，可能升级变更里能找到信息，不过我们以学习为目的，可以看看代码是怎么样的
报错是在这段代码里的
org.apache.tomcat.util.http.fileupload.util.LimitedInputStream#checkLimit

1
2
3
4
5
private void checkLimit() throws IOException {
    if (count > sizeMax) {
        raiseError(sizeMax, count);
    }
}

其中的 raiseError 是个抽象方法

1
2
protected abstract void raiseError(long pSizeMax, long pCount)
            throws IOException;

具体的实现是在

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
public FileItemStreamImpl(FileItemIteratorImpl pFileItemIterator, String pName, String pFieldName, String pContentType, boolean pFormField, long pContentLength) throws FileUploadException, IOException {
        this.fileItemIteratorImpl = pFileItemIterator;
        this.name = pName;
        this.fieldName = pFieldName;
        this.contentType = pContentType;
        this.formField = pFormField;
        long fileSizeMax = this.fileItemIteratorImpl.getFileSizeMax();
        if (fileSizeMax != -1L && pContentLength != -1L && pContentLength > fileSizeMax) {
            FileSizeLimitExceededException e = new FileSizeLimitExceededException(String.format("The field %s exceeds its maximum permitted size of %s bytes.", this.fieldName, fileSizeMax), pContentLength, fileSizeMax);
            e.setFileName(pName);
            e.setFieldName(pFieldName);
            throw new FileUploadIOException(e);
        } else {
            final MultipartStream.ItemInputStream itemStream = this.fileItemIteratorImpl.getMultiPartStream().newInputStream();
            InputStream istream = itemStream;
            if (fileSizeMax != -1L) {
                istream = new LimitedInputStream(itemStream, fileSizeMax) {
                    protected void raiseError(long pSizeMax, long pCount) throws IOException {
                        itemStream.close(true);
                        FileSizeLimitExceededException e = new FileSizeLimitExceededException(String.format("The field %s exceeds its maximum permitted size of %s bytes.", FileItemStreamImpl.this.fieldName, pSizeMax), pCount, pSizeMax);
                        e.setFieldName(FileItemStreamImpl.this.fieldName);
                        e.setFileName(FileItemStreamImpl.this.name);
                        throw new FileUploadIOException(e);
                    }
                };
            }

            this.stream = (InputStream)istream;
        }
    }

后面也会介绍到，这里我们其实主要是要找到这个 pSizeMax 是哪里来的
通过阅读代码会发现跟这个类 MultipartConfigElement 有关系
而在升级后的 springboot 中这个类已经有了自动装配类，也就是
org.springframework.boot.autoconfigure.web.servlet.MultipartAutoConfiguration

有了这个自动装配

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
@Configuration(
    proxyBeanMethods = false
)
@ConditionalOnClass({Servlet.class, StandardServletMultipartResolver.class, MultipartConfigElement.class})
@ConditionalOnProperty(
    prefix = "spring.servlet.multipart",
    name = {"enabled"},
    matchIfMissing = true
)
@ConditionalOnWebApplication(
    type = Type.SERVLET
)
@EnableConfigurationProperties({MultipartProperties.class})
public class MultipartAutoConfiguration {
    private final MultipartProperties multipartProperties;

    public MultipartAutoConfiguration(MultipartProperties multipartProperties) {
        this.multipartProperties = multipartProperties;
    }

    @Bean
    @ConditionalOnMissingBean({MultipartConfigElement.class, CommonsMultipartResolver.class})
    public MultipartConfigElement multipartConfigElement() {
        return this.multipartProperties.createMultipartConfig();
    }

而这个 MultipartProperties 类中

1
2
3
4
5
6
7
8
9
10
11
@ConfigurationProperties(
    prefix = "spring.servlet.multipart",
    ignoreUnknownFields = false
)
public class MultipartProperties {
    private boolean enabled = true;
    private String location;
    private DataSize maxFileSize = DataSize.ofMegabytes(1L);
    private DataSize maxRequestSize = DataSize.ofMegabytes(10L);
    private DataSize fileSizeThreshold = DataSize.ofBytes(0L);
    private boolean resolveLazily = false;

并且在前面 createMultipartConfig 中就使用了这个maxFileSize 的默认值

1
2
3
4
5
6
7
8
9
public MultipartConfigElement createMultipartConfig() {
    MultipartConfigFactory factory = new MultipartConfigFactory();
    PropertyMapper map = PropertyMapper.get().alwaysApplyingWhenNonNull();
    map.from(this.fileSizeThreshold).to(factory::setFileSizeThreshold);
    map.from(this.location).whenHasText().to(factory::setLocation);
    map.from(this.maxRequestSize).to(factory::setMaxRequestSize);
    map.from(this.maxFileSize).to(factory::setMaxFileSize);
    return factory.createMultipartConfig();
}

而在 org.apache.catalina.connector.Request#parseParts 中，会判断 mce 的配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
private void parseParts(boolean explicit) {

        // 省略一部分代码

            ServletFileUpload upload = new ServletFileUpload();
            upload.setFileItemFactory(factory);
            upload.setFileSizeMax(mce.getMaxFileSize());
            upload.setSizeMax(mce.getMaxRequestSize());

            parts = new ArrayList<>();
            try {
                List<FileItem> items =
                        upload.parseRequest(new ServletRequestContext(this));
                int maxPostSize = getConnector().getMaxPostSize();
                int postSize = 0;
                Charset charset = getCharset();

主要 org.apache.tomcat.util.http.fileupload.FileUploadBase#parseRequest

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
public List<FileItem> parseRequest(final RequestContext ctx)
            throws FileUploadException {
        final List<FileItem> items = new ArrayList<>();
        boolean successful = false;
        try {
            final FileItemIterator iter = getItemIterator(ctx);
            final FileItemFactory fileItemFactory = Objects.requireNonNull(getFileItemFactory(),
                    "No FileItemFactory has been set.");
            final byte[] buffer = new byte[Streams.DEFAULT_BUFFER_SIZE];
            while (iter.hasNext()) {
                final FileItemStream item = iter.next();
                // Don't use getName() here to prevent an InvalidFileNameException.
                final String fileName = item.getName();
                final FileItem fileItem = fileItemFactory.createItem(item.getFieldName(), item.getContentType(),
                                                   item.isFormField(), fileName);
                items.add(fileItem);
                try {
                    Streams.copy(item.openStream(), fileItem.getOutputStream(), true, buffer);
                } catch (final FileUploadIOException e) {

其中 org.apache.tomcat.util.http.fileupload.FileUploadBase#getItemIterator

1
2
3
4
5
6
7
8
9
public FileItemIterator getItemIterator(final RequestContext ctx)
throws FileUploadException, IOException {
    try {
        return new FileItemIteratorImpl(this, ctx);
    } catch (final FileUploadIOException e) {
        // unwrap encapsulated SizeException
        throw (FileUploadException) e.getCause();
    }
}

这里就创建了 org.apache.tomcat.util.http.fileupload.impl.FileItemIteratorImpl

1
2
3
4
5
6
7
8
9
10
public FileItemIteratorImpl(final FileUploadBase fileUploadBase, final RequestContext requestContext)
    throws FileUploadException, IOException {
    this.fileUploadBase = fileUploadBase;
    sizeMax = fileUploadBase.getSizeMax();
    fileSizeMax = fileUploadBase.getFileSizeMax();
    ctx = Objects.requireNonNull(requestContext, "requestContext");
    skipPreamble = true;
    findNextItem();
}

内部使用了前面给 upload 设置的文件大小上限 upload.setFileSizeMax(mce.getMaxFileSize());

然后在 findNextItem 里执行了初始化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
private boolean findNextItem() throws FileUploadException, IOException {
        if (eof) {
            return false;
        }
        if (currentItem != null) {
            currentItem.close();
            currentItem = null;
        }
        final MultipartStream multi = getMultiPartStream();
        for (;;) {
            final boolean nextPart;
            if (skipPreamble) {
                nextPart = multi.skipPreamble();
            } else {
                nextPart = multi.readBoundary();
            }
            if (!nextPart) {
                if (currentFieldName == null) {
                    // Outer multipart terminated -> No more data
                    eof = true;
                    return false;
                }
                // Inner multipart terminated -> Return to parsing the outer
                multi.setBoundary(multiPartBoundary);
                currentFieldName = null;
                continue;
            }
            final FileItemHeaders headers = fileUploadBase.getParsedHeaders(multi.readHeaders());
            if (currentFieldName == null) {
                // We're parsing the outer multipart
                final String fieldName = fileUploadBase.getFieldName(headers);
                if (fieldName != null) {
                    final String subContentType = headers.getHeader(FileUploadBase.CONTENT_TYPE);
                    if (subContentType != null
                            &&  subContentType.toLowerCase(Locale.ENGLISH)
                                    .startsWith(FileUploadBase.MULTIPART_MIXED)) {
                        currentFieldName = fieldName;
                        // Multiple files associated with this field name
                        final byte[] subBoundary = fileUploadBase.getBoundary(subContentType);
                        multi.setBoundary(subBoundary);
                        skipPreamble = true;
                        continue;
                    }
                    final String fileName = fileUploadBase.getFileName(headers);
                    currentItem = new FileItemStreamImpl(this, fileName,
                            fieldName, headers.getHeader(FileUploadBase.CONTENT_TYPE),
                            fileName == null, getContentLength(headers));
                    currentItem.setHeaders(headers);
                    progressNotifier.noteItem();
                    itemValid = true;
                    return true;
                }
            } else {
                final String fileName = fileUploadBase.getFileName(headers);
                if (fileName != null) {
                    currentItem = new FileItemStreamImpl(this, fileName,
                            currentFieldName,
                            headers.getHeader(FileUploadBase.CONTENT_TYPE),
                            false, getContentLength(headers));
                    currentItem.setHeaders(headers);
                    progressNotifier.noteItem();
                    itemValid = true;
                    return true;
                }
            }
            multi.discardBodyData();
        }
    }

这里面就会 new 这个 FileItemStreamImpl

1
2
3
4
currentItem = new FileItemStreamImpl(this, fileName,
                            fieldName, headers.getHeader(FileUploadBase.CONTENT_TYPE),
                            fileName == null, getContentLength(headers));

构造方法比较长

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
public FileItemStreamImpl(final FileItemIteratorImpl pFileItemIterator, final String pName, final String pFieldName,
            final String pContentType, final boolean pFormField,
            final long pContentLength) throws FileUploadException, IOException {
        fileItemIteratorImpl = pFileItemIterator;
        name = pName;
        fieldName = pFieldName;
        contentType = pContentType;
        formField = pFormField;
        final long fileSizeMax = fileItemIteratorImpl.getFileSizeMax();
        if (fileSizeMax != -1 && pContentLength != -1
                && pContentLength > fileSizeMax) {
            final FileSizeLimitExceededException e =
                    new FileSizeLimitExceededException(
                            String.format("The field %s exceeds its maximum permitted size of %s bytes.",
                                    fieldName, Long.valueOf(fileSizeMax)),
                            pContentLength, fileSizeMax);
            e.setFileName(pName);
            e.setFieldName(pFieldName);
            throw new FileUploadIOException(e);
        }
        // OK to construct stream now
        final ItemInputStream itemStream = fileItemIteratorImpl.getMultiPartStream().newInputStream();
        InputStream istream = itemStream;
        if (fileSizeMax != -1) {
            istream = new LimitedInputStream(istream, fileSizeMax) {
                @Override
                protected void raiseError(final long pSizeMax, final long pCount)
                        throws IOException {
                    itemStream.close(true);
                    final FileSizeLimitExceededException e =
                        new FileSizeLimitExceededException(
                            String.format("The field %s exceeds its maximum permitted size of %s bytes.",
                                   fieldName, Long.valueOf(pSizeMax)),
                            pCount, pSizeMax);
                    e.setFieldName(fieldName);
                    e.setFileName(name);
                    throw new FileUploadIOException(e);
                }
            };
        }
        stream = istream;
    }

当 fileSizeMax != 0 的时候就会初始化 LimitedInputStream，这就就是会在前面的

org.apache.tomcat.util.http.fileupload.FileUploadBase#parseRequest 中

1
Streams.copy(item.openStream(), fileItem.getOutputStream(), true, buffer);

这里的 item

1
2
3
4
5
6
final FileItemIterator iter = getItemIterator(ctx);
            final FileItemFactory fileItemFactory = Objects.requireNonNull(getFileItemFactory(),
                    "No FileItemFactory has been set.");
            final byte[] buffer = new byte[Streams.DEFAULT_BUFFER_SIZE];
            while (iter.hasNext()) {
                final FileItemStream item = iter.next();

调用了 FileItemIterator 迭代器的 next

1
2
3
4
5
6
7
8
@Override
    public FileItemStream next() throws FileUploadException, IOException {
        if (eof  ||  (!itemValid && !hasNext())) {
            throw new NoSuchElementException();
        }
        itemValid = false;
        return currentItem;
    }

这个 currentItem 就是前面 new 的 FileItemStreamImpl

然后在 Streams.copy 的时候调用 openStream 也就是 org.apache.tomcat.util.http.fileupload.impl.FileItemStreamImpl#openStream

1
2
3
4
5
6
7
@Override
    public InputStream openStream() throws IOException {
        if (((Closeable) stream).isClosed()) {
            throw new FileItemStream.ItemSkippedException();
        }
        return stream;
    }

这里的 stream 就是 FileItemStreamImpl 构造方法最后赋值的 stream，会在大小超过限制时抛出错误

而这个可以通过设置 properties 来修改，spring.servlet.multipart.max-file-size 和 spring.servlet.multipart.max-request-size

1
2
spring.servlet.multipart.max-file-size=100MB
spring.servlet.multipart.max-request-size=100MB

而老版本的 spring.http.multipart.maxFileSize
其实就是配置名称改了下，但是能看一下代码也是有点收获的。