HttpClient 多线程请求


写一段客户端测试的代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
public class HttpClient {

private static final Logger LOGGER = LoggerFactory.getLogger(HttpClient.class);

private static final CloseableHttpClient staticClient;
static {
RequestConfig requestConfig = RequestConfig.custom()
.setSocketTimeout(8000)
.setConnectTimeout(8000)
.build();


staticClient = HttpClients.custom()
.setDefaultRequestConfig(requestConfig)
.useSystemProperties()
.build();
}


public static void main(String[] args) throws InterruptedException {

TimeUnit.SECONDS.sleep(10);

for (int i = 1; i < 20; i++) {
startRequest(i);
}
}


private static void startRequest(int idx) {
new Thread(() -> {

HttpHost httpHost = new HttpHost("localhost", 8080);
HttpRequest httpRequest = new HttpGet("/" + idx);
try {
CloseableHttpClient httpClient = getClient();
print(httpClient);


LOGGER.info("请求开始: " + idx);
httpClient.execute(httpHost, httpRequest);
LOGGER.info("请求结束: " + idx);
} catch (IOException e) {
LOGGER.error("请求异常: " + idx, e);
}
}).start();
}


private static CloseableHttpClient getClient() {
return staticClient;
}

private static void print(CloseableHttpClient httpClient) {
try {

Field field = httpClient.getClass().getDeclaredField("connManager");
field.setAccessible(true);

PoolingHttpClientConnectionManager clientConnectionManager = (PoolingHttpClientConnectionManager)field.get(httpClient);
PoolStats stats = clientConnectionManager.getTotalStats();
System.out.println("Max : " + stats.getMax()
+ ". Available : " + stats.getAvailable()
+ ". Leased : " + stats.getLeased()
+ ". Pending : " + stats.getPending()
+ ". DefaultMaxPerRoute : " + clientConnectionManager.getDefaultMaxPerRoute()
+ ". MaxTotal : " + clientConnectionManager.getMaxTotal()
);

field.setAccessible(false);
} catch (Exception e) {
e.printStackTrace();
}
}
}

服务端沉睡3秒钟,每次执行这段代码的话,都只会有2个请求执行完成。
这是因为在HttpClient的连接池中MaxtTotalDefaultMaxPerRoute有这么俩个参数,

  • MaxtTotal 表示连接池中最大的连接数,
  • DefaultMaxPerRoute 表示某个地址最大的并发连接数。

比如MaxtTotal为20,DefaultMaxPerRoute为2(这俩个值也是默认值)。
现在有10个并发请求,其中4个请求www.baidu.com, 6个请求www.qq.com, 如果服务器都阻塞住的话,那么现在的并发连接是4个,baidu2个,qq俩个。
如果在RequestConfig#setConnectionRequestTimeout这个没有设置的话,其他的请求线程就会一直阻塞了。

解决这个问题的话,有三种方案

  1. 设置RequestConfig#setConnectionRequestTimeout这个值,大于0的话,如果到达时间没有可用连接的话,就会抛出异常
  2. 设置System.setProperty("http.maxConnections", "20”); 这个值会调大 DefaultMaxPerRoute 这个值
  3. 不要共享HttpClient,如果想要降低new的话,可以使用ThreadLocal