็†ฑ้–€ๅˆ†้กž
 ่ผ‰ๅ…ฅไธญ…
็›ฎ้Œ„

๐Ÿš€ Linux ๆ•ˆ่ƒฝ่ชฟๅ„ช:้‡ๅฐ้ซ˜ไฝต็™ผ Nginx ็’ฐๅขƒ็š„ Sysctl ๆ ธๅฟƒๅƒๆ•ธๅ„ชๅŒ–ๅ…จ็ด€้Œ„

    ๐Ÿš€ Linux ๆ•ˆ่ƒฝ่ชฟๅ„ช:้‡ๅฐ้ซ˜ไฝต็™ผ Nginx ็’ฐๅขƒ็š„ Sysctl ๆ ธๅฟƒๅƒๆ•ธๅ„ชๅŒ–ๅ…จ็ด€้Œ„


    ไธ€、้€™็ฏ‡ๅœจ่งฃๆฑบไป€้บผๅ•้กŒ?(้ซ˜ไฝต็™ผ Nginx ็š„ๅ…ธๅž‹็“ถ้ ธ)

    Nginx ๆœฌ่บซๅพˆ่ƒฝๆ‰›,ไฝ†「ๆ‰›ๅพ—ไฝ」ไธไปฃ่กจ「ๆ‰›ๅพ—ๆผ‚ไบฎ」。้ซ˜ไฝต็™ผไธ‹ๅธธ่ฆ‹ๅก้ปžไธๅœจ Nginx,่€Œๅœจ Linux ๆ ธๅฟƒ็š„่ณ‡ๆบไธŠ้™่ˆ‡ TCP/IP ไฝ‡ๅˆ—:

    • ้€ฃ็ทšไฝ‡ๅˆ—ๅคชๅฐ:accept queue / SYN backlog ๆ’ไธไฝ,ๅ‡บ็พไธŸๅŒ…、้‡ๅ‚ณ、ๆกๆ‰‹ๅปถ้ฒ。
    • TIME_WAIT/็Ÿญ้€ฃ็ทšๅฃ“ๅŠ›:ๅคง้‡็Ÿญ้€ฃ็ทš้€ ๆˆ port、socket、่จ˜ๆ†ถ้ซ”่ˆ‡ๅ›žๆ”ถๅฃ“ๅŠ›。
    • socket buffer ไธๅค :ๅžๅไธŠไธๅŽป、ๅปถ้ฒๆŠ–ๅ‹•、ๅฐ–ๅณฐๆ™‚ๆ›ดๆ˜Ž้กฏ。
    • ๆช”ๆกˆๆ่ฟฐ็ฌฆไธๅค :็›ดๆŽฅๅ™ด Too many open files ๆˆ–้šฑๆ€งๆމ้€ฃ็ทš。
    • NIC/backlog ็ทฉ่กไธ่ถณ:ๅฐๅŒ…ไธ€ๅคšๅฐฑๆŽ’้šŠ、drop。

    ไบŒ、ๅ…ˆๅปบ็ซ‹「ๅŸบๆบ–」:ไธ่ฆไธ€ไธŠไพ†ๅฐฑไบ‚ๆ”น

    ๅปบ่ญฐๅ…ˆๆŠŠ็พๆณๆŠ“ไธ‹ไพ†,ๅพŒ้ข่ชฟๆ•ดๆ‰็Ÿฅ้“ๆ”นไบ†ไป€้บผ、ๆœ‰ๆ•ˆๆฒ’ๆ•ˆ。

    # 1) ๆ ธๅฟƒ็‰ˆๆœฌ / CPU / ่จ˜ๆ†ถ้ซ” / NIC
    uname -a
    lscpu
    free -h
    ip -s link
    
    # 2) ็›ฎๅ‰ sysctl(ๆŠ“้‡้ปžๅณๅฏ)
    sysctl net.core.somaxconn
    sysctl net.core.netdev_max_backlog
    sysctl net.ipv4.tcp_max_syn_backlog
    sysctl net.ipv4.ip_local_port_range
    sysctl net.ipv4.tcp_fin_timeout
    sysctl net.ipv4.tcp_tw_reuse
    sysctl net.ipv4.tcp_keepalive_time
    sysctl net.ipv4.tcp_congestion_control
    sysctl net.ipv4.tcp_fastopen
    
    # 3) ้€ฃ็ทš่ˆ‡ socket ๆฆ‚ๆณ(ๅฐ–ๅณฐๆ™‚ๆœ€ๅฅฝไนŸๆŠ“ไธ€ๆฌก)
    ss -s
    cat /proc/net/sockstat
    cat /proc/net/sockstat6
    
    # 4) Nginx process ๆช”ๆกˆไธŠ้™(ๅพˆ้‡่ฆ)
    pidof nginx | tr ' ' '\n' | head -n 1 | xargs -I{} cat /proc/{}/limits | grep -i "Max open files"

    ไธ‰、ๆŽจ่–ฆ Sysctl ้…็ฝฎ(้ซ˜ไฝต็™ผ Web / Reverse Proxy ๅธธ็”จ็ต„ๅˆ)

    ไปฅไธ‹ๆไพ›ไธ€ไปฝ「ๅไฟๅฎˆ、ๅฏ็”จๆ–ผๅคšๆ•ธ้ซ˜ไฝต็™ผ Nginx」็š„่ตทๆ‰‹ๅผ。ไธๆ˜ฏ่ถŠๅคง่ถŠๅฅฝ,่€Œๆ˜ฏ่ฆ่ทŸไฝ ็š„ๆต้‡ๅž‹ๆ…‹(้•ท้€ฃ็ทš/็Ÿญ้€ฃ็ทš)、ไธŠๆธธๆœๅ‹™、ไปฅๅŠ NIC ่ƒฝๅŠ›ๅŒน้…。

    # ๅปบ่ญฐๅปบ็ซ‹็จ็ซ‹ๆช”ๆกˆ,ไพฟๆ–ผๆŽง็‰ˆ่ˆ‡ๅ›žๆปพ
    sudo tee /etc/sysctl.d/99-nginx-highconcurrency.conf > /dev/null <<'EOF'
    # =========================================================
    # Nginx High Concurrency Sysctl Tuning
    # ็›ฎ็š„:ๆๅ‡ไฝ‡ๅˆ—ๆ‰ฟ่ผ‰、้™ไฝŽๅฐ–ๅณฐ drop、ๆ”นๅ–„็Ÿญ้€ฃ็ทšๅฃ“ๅŠ›
    # ๆณจๆ„:ๅฅ—็”จๅ‰่ซ‹ๅ…ˆๅปบ็ซ‹ๅŸบๆบ–ไธฆๆบ–ๅ‚™ๅ›žๆปพ
    # =========================================================
    
    # --- 1) ้€ฃ็ทšไฝ‡ๅˆ— / ็ถฒๅก backlog ---
    net.core.somaxconn = 4096
    net.core.netdev_max_backlog = 16384
    
    # --- 2) TCP ๆกๆ‰‹ / SYN backlog(ๅŠ้€ฃ็ทš)---
    net.ipv4.tcp_max_syn_backlog = 8192
    net.ipv4.tcp_syncookies = 1
    
    # --- 3) ephemeral ports(ๅคง้‡็Ÿญ้€ฃ็ทš、ๅๅ‘ไปฃ็†็‰นๅˆฅๅธธ่ฆ‹)---
    net.ipv4.ip_local_port_range = 10240 65535
    
    # --- 4) TIME_WAIT / ้€ฃ็ทšๅ›žๆ”ถ(ไฟๅฎˆ่จญๅฎš:ไธๆŠŠๅฎƒ็•ถ้ญ”ๆณ•)---
    # tcp_tw_reuse:ๅๅ‘ client-side,ๅคง้‡็Ÿญ้€ฃ็ทšๆ™‚ๅฏ่ƒฝๆœ‰ๅŠฉ,ไฝ†ไธๆ˜ฏ้€š็”จ่งฃๆณ•
    net.ipv4.tcp_tw_reuse = 0
    net.ipv4.tcp_fin_timeout = 30
    
    # --- 5) keepalive(้ฟๅ…ๆญป้€ฃ็ทšไฝ”่ณ‡ๆบ;ไพๆœๅ‹™ๅž‹ๆ…‹่ชฟๆ•ด)---
    net.ipv4.tcp_keepalive_time = 600
    net.ipv4.tcp_keepalive_intvl = 30
    net.ipv4.tcp_keepalive_probes = 5
    
    # --- 6) socket buffer(ๅžๅ่ˆ‡ๅปถ้ฒ็š„ๅŸบ็คŽ็›ค;ไธ่ฆ็„ก่…ฆๆ‹‰็ˆ†)---
    net.core.rmem_max = 134217728
    net.core.wmem_max = 134217728
    net.ipv4.tcp_rmem = 4096 1048576 134217728
    net.ipv4.tcp_wmem = 4096 1048576 134217728
    
    # --- 7) ๅ…ถไป–ๅธธ่ฆ‹็ฉฉๅฎš้ …(็ถญๆŒ้ ่จญๅณๅฏ,ไนŸๅฏๆ˜Ž็ขบๅฏซๅ‡บ)---
    net.ipv4.tcp_timestamps = 1
    net.ipv4.tcp_sack = 1
    net.ipv4.tcp_window_scaling = 1
    
    # --- 8) FQ + BBR(่‹ฅๆ ธๅฟƒ่ˆ‡็™ผ่กŒ็‰ˆๆ”ฏๆด,ๅฏ่€ƒๆ…ฎ;ไธๅผทๅˆถ)---
    # net.core.default_qdisc = fq
    # net.ipv4.tcp_congestion_control = bbr
    
    # --- 9) ๆช”ๆกˆๆ่ฟฐ็ฌฆ(็ณป็ตฑๅฑคไธŠ้™;ไป้œ€้…ๅˆ systemd / nginx ่จญๅฎš)---
    fs.file-max = 2097152
    EOF
    
    # ๅฅ—็”จ
    sudo sysctl --system
    
    # ้ฉ—่ญ‰
    sysctl net.core.somaxconn
    sysctl net.ipv4.tcp_max_syn_backlog
    sysctl net.ipv4.ip_local_port_range

    ๅ››、ๆฏๅ€‹ๅƒๆ•ธๅˆฐๅบ•ๅœจ่ชฟไป€้บผ?(็™ฝ่ฉฑไฝ†ไธๅซ็ณŠ)

    1) net.core.somaxconn:accept queue ไธŠ้™

    ็•ถ้€ฃ็ทšๅฎŒๆˆไธ‰ๅ‘ไบคๆกๅพŒ,ๆœƒ้€ฒๅ…ฅ็ญ‰ๅพ… accept() ็š„ไฝ‡ๅˆ—。้€™ๅ€‹ไฝ‡ๅˆ—ๅคชๅฐ,้ซ˜ๅณฐๅฐฑๆœƒๆŽ’ไธ้€ฒๅŽป。Nginx ็š„ listen backlog ไธๆ‡‰้ซ˜ๆ–ผ somaxconn,ๅฆๅ‰‡ไนŸๆœƒ่ขซๆˆชๆ–ท。

    2) net.ipv4.tcp_max_syn_backlog:SYN backlog(ๅŠ้€ฃ็ทšไธŠ้™)

    ๆกๆ‰‹ๆœชๅฎŒๆˆๅ‰็š„ๅŠ้€ฃ็ทšไฝ‡ๅˆ—。ๅฐ–ๅณฐๆˆ–้‡ๅˆฐ SYN flood ๆ™‚,้€™่ฃกๅคชๅฐๅฐฑๅฎนๆ˜“ๆމๆกๆ‰‹。tcp_syncookies=1 ๅฏไฝœ็‚บๅฃ“ๅŠ›ไฟ่ญท,ไฝ†ไธๅปบ่ญฐๆŠŠๅฎƒ็•ถไฝœ้•ทๆœŸไพ่ณด,ไป่ฆๆญ้…ๅˆ็†็š„ไฝ‡ๅˆ—่ˆ‡้˜ฒ่ญท。

    3) net.core.netdev_max_backlog:NIC ๆ”ถๅŒ…ๅˆฐ CPU ๅ‰็š„็ทฉ่ก

    ็ถฒๅก้€ฒไพ†็š„ๅฐๅŒ…,CPU ๅฟ™ๆ™‚ๆœƒๅ…ˆๆŽ’้šŠ。้€™่ฃกๅๅฐๅฐฑๅฎนๆ˜“ drop,ๅฐ–ๅณฐๆŠ–ๅ‹•ๆœƒ่ฎŠๅคง。ๅธธ่ฆ‹ๅšๆณ•ๆ˜ฏ้…ๅˆไธญๆ–ท็ถๆ ธ、RPS/RFS、ไปฅๅŠ NIC ring buffer ไธ€่ตท็œ‹,ไฝ†้€™็ฏ‡ๅ…ˆ่š็„ฆ sysctl。

    4) net.ipv4.ip_local_port_range:ๆœฌๆฉŸ่‡จๆ™‚ port ็ฏ„ๅœ

    ๅๅ‘ไปฃ็†/ไธŠๆธธ้€ฃ็ทšๅคšๆ™‚,Nginx ๆœƒๅฐไธŠๆธธๅปบ็ซ‹ๅคง้‡ๆœฌๆฉŸ้€ฃ็ทš,ๆœƒ่€—ๆމ ephemeral ports。็ฏ„ๅœๅคช็ช„ๆœƒๅ‡บ็พ「้€ฃ็ทš็ช็„ถไธŠไธๅŽป」ๆˆ–ๅถ็™ผ้Œฏ่ชค。ๆŠŠ็ฏ„ๅœๆ‹‰ๅคง้€šๅธธๆ˜ฏไฝŽ้ขจ้šช้ซ˜ๆ”ถ็›Š็š„ไธ€ๆญฅ。

    5) tcp_fin_timeout / tcp_tw_reuse:็Ÿญ้€ฃ็ทš็š„ๅœฐ้›ทๅ€

    ๅพˆๅคšไบบ็œ‹ๅˆฐ TIME_WAIT ๅฐฑๆƒณ「ๆถˆๆป…ๅฎƒ」,ไฝ† TIME_WAIT ๆ˜ฏ TCP ๆญฃๅธธๆฉŸๅˆถ็š„ไธ€้ƒจๅˆ†。
    ๆœฌ็ฏ‡ๆŽกไฟๅฎˆ็ญ–็•ฅ:ๅ…ˆไธๆŠŠ tcp_tw_reuse ็•ถๆˆ้€š็”จ่งฃๆณ•,ๅชๅšๆœ‰้™็š„ๅ›žๆ”ถ้€Ÿๅบฆ่ชฟๆ•ด(ไพ‹ๅฆ‚ tcp_fin_timeout)。ๅฆ‚ๆžœไฝ ็š„ๆƒ…ๅขƒๆ˜ฏ「ๅคง้‡ client-side ็Ÿญ้€ฃ็ทš」ๆˆ–「loopback ๅฐๅŒ…」็ญ‰็‰นๆฎŠๆกˆไพ‹,ๅ†ๅฆๅค–ๅš A/B ้ฉ—่ญ‰ๆœƒๆฏ”่ผƒๅฎ‰ๅ…จ。

    6) socket buffer:ๅžๅ็š„ๅบ•ๅบง,ไฝ†ไนŸๅฏ่ƒฝๅƒๅ…‰่จ˜ๆ†ถ้ซ”

    ๆŠŠ rmem/wmem ๆ‹‰้ซ˜,ๆœ‰ๅŠฉๆ–ผ้ซ˜ๅžๅ、้•ท RTT ๆˆ–็ช็™ผๆต้‡,ไฝ†ไนŸๆœƒๅขžๅŠ ่จ˜ๆ†ถ้ซ”ๅฃ“ๅŠ›。ๅปบ่ญฐ็”จ็›ฃๆŽง็œ‹ socket memory、TCP retrans、ไปฅๅŠ p99 latency,ๅ†้€ๆญฅ่ชฟๆ•ด,ไธ่ฆไธ€ๆฌกๆ‹‰ๅˆฐ่ช‡ๅผต。


    ไบ”、ไธ่ฆๅช่ชฟ Sysctl:Nginx ่ˆ‡ systemd ๅฟ…้ ˆๅŒๆญฅ

    Sysctl ๆŠŠ「้“่ทฏๆ‹“ๅฏฌ」ไบ†,ไฝ† Nginx ้€™ๅฐ่ปŠๅฆ‚ๆžœ้™้€Ÿ(nofile / worker_connections)ๆฒ’่งฃ้–‹,ๆ•ˆๆžœๆœƒ็›ดๆŽฅ่ขซๅกๆญป。

    1) systemd:ๆๅ‡ Nginx ๆœๅ‹™็š„ nofile ไธŠ้™

    # ๅปบ็ซ‹ override(ไธๅŒ็™ผ่กŒ็‰ˆ nginx.service ไฝ็ฝฎไธๅŒ,ไฝ†ๅšๆณ•ไธ€่‡ด)
    sudo systemctl edit nginx.service
    
    # ๅŠ ไธŠ:
    # [Service]
    # LimitNOFILE=1048576
    
    sudo systemctl daemon-reload
    sudo systemctl restart nginx
    
    # ้ฉ—่ญ‰
    pidof nginx | tr ' ' '\n' | head -n 1 | xargs -I{} cat /proc/{}/limits | grep -i "Max open files"

    2) Nginx:worker ้€ฃ็ทš่ˆ‡ๆช”ๆกˆๆ่ฟฐ็ฌฆ

    # /etc/nginx/nginx.conf(็คบๆ„)
    worker_processes auto;
    worker_rlimit_nofile 1048576;
    
    events {
      worker_connections  65535;
      multi_accept on;
      use epoll;
    }
    
    # ่‹ฅไฝ ๆœ‰ๆ˜Ž็ขบ้ซ˜ๅณฐๆŽ’้šŠๅ•้กŒ,ๅฏ่€ƒๆ…ฎๆ˜Ž็ขบๆŒ‡ๅฎš backlog(้œ€่ˆ‡ somaxconn ๅฐ้ฝŠ)
    # server {
    #   listen 443 ssl http2 backlog=4096;
    # }

    ๅ…ญ、้ฉ—่ญ‰ๆ˜ฏๅฆ็œŸ็š„「่ฎŠๅฟซไบ†」:็œ‹้€™ไบ›ๅฐฑๅค ็”จ

    ่ชฟๆ กๆˆๅŠŸ้€šๅธธไธๆ˜ฏ「CPU ้™ๅพˆๅคš」้€™้บผ็›ด่ฆบ,่€Œๆ˜ฏๅฐ–ๅณฐๆމๅŒ…ไธ‹้™、ๆกๆ‰‹ๆ›ด็ฉฉ、p95/p99 latency ๆ”ถๆ–‚、้€ฃ็ทš้Œฏ่ชคไธ‹้™。

    # 1) ้€ฃ็ทš็ธฝ่ฆฝ
    ss -s
    
    # 2) SYN / listen overflow / drop(ๆœ‰ไบ›่จˆๆ•ธๅฏๅพžๅ…งๆ ธ็ตฑ่จˆ่ง€ๅฏŸ)
    netstat -s | egrep -i "listen|overflow|drop|retrans|syn"
    
    # 3) socket ่ณ‡ๆบ
    cat /proc/net/sockstat
    cat /proc/net/sockstat6
    
    # 4) Nginx ็‹€ๆ…‹(่‹ฅไฝ ๆœ‰ๅ•Ÿ็”จ stub_status)
    # curl -s http://127.0.0.1/nginx_status
    
    # 5) ๅฃ“ๆธฌ(็ฏ„ไพ‹:่ซ‹ไพไฝ ็’ฐๅขƒๆŒ‘ๅทฅๅ…ท)
    # wrk -t8 -c2000 -d60s https://your-domain/
    # hey -z 60s -c 2000 https://your-domain/

    ไธƒ、ๅ›žๆปพ็ญ–็•ฅ(ไธ€ๅฎš่ฆๅ…ˆๆบ–ๅ‚™)

    ไธ€ๆ—ฆ็™ผ็พ็•ฐๅธธ(ไพ‹ๅฆ‚้€ฃ็ทš่กŒ็‚บๆ”น่ฎŠ、ไธŠๆธธ timeout、p99 ่ฎŠๅทฎ),ๅ›žๆปพ่ฆ่ƒฝๅœจ 1 ๅˆ†้˜ๅ…งๅฎŒๆˆ。

    # 1) ็›ดๆŽฅ็งป้™ค้…็ฝฎๆช”ไธฆ้‡่ผ‰ sysctl
    sudo mv /etc/sysctl.d/99-nginx-highconcurrency.conf /root/99-nginx-highconcurrency.conf.bak
    sudo sysctl --system
    
    # 2) ๆˆ–ๅชๅ›ž้€€ๅ–ฎไธ€ๅƒๆ•ธ(็ซ‹ๅณ็”Ÿๆ•ˆไฝ†ไธๆŒไน…)
    sudo sysctl -w net.core.somaxconn=128
    
    # 3) ็ขบ่ชๅ€ผ
    sysctl net.core.somaxconn

    ๅ…ซ、ๅธธ่ฆ‹้™ท้˜ฑ่ˆ‡ๅ‹™ๅฏฆๅปบ่ญฐ

    • 「ๅ€ผ่ถŠๅคง่ถŠๅฅฝ」ๆ˜ฏ้Œฏ็š„:ไฝ‡ๅˆ—่ˆ‡ buffer ๆœƒๅƒ่ณ‡ๆบ,ๅฐ–ๅณฐๅฅฝ็œ‹、ๅธธๆ…‹่ฎŠๆ…ขไนŸไธๆ„ๅค–。
    • TIME_WAIT ไธ็ญ‰ๆ–ผๅ•้กŒ:ๅฎƒๅธธๆ˜ฏ「ไฝ ็œŸ็š„ๅพˆๅฟ™」็š„ๅ‰ฏไฝœ็”จ;ๅ…ˆ็ขบ่ชๆ˜ฏๅฆๆ˜ฏ็Ÿญ้€ฃ็ทš้Žๅคš、Keep-Alive ่จญ่จˆไธๅˆ็†。
    • ๅๅ‘ไปฃ็†่ฆ็œ‹ ephemeral ports:ไธŠๆธธ้€ฃ็ทšๅคšๆ™‚,ip_local_port_range ๅพ€ๅพ€ๆฏ”ไฝ ๆƒณๅพ—ๆ›ด้‡่ฆ。
    • Gateway、DNS、ไธŠๆธธ RTT:็ถฒ่ทฏ่ทฏๅพ‘(Gateway)、DNS ๅ›žๆ‡‰่ˆ‡ไธŠๆธธๅปถ้ฒ,ๅธธๅธธๆ‰ๆ˜ฏ p99 ็š„็œŸๆญฃๅ…‡ๆ‰‹,ๅˆฅๅช็›ฏ sysctl。
    • ๆœ€ๆœ‰ๆ•ˆ็š„ๆ–นๅผ:ๅ…ˆๅปบ็ซ‹ๅŸบๆบ– → ๅฐๆญฅๆ”นๅ‹• → ็œ‹ๆŒ‡ๆจ™ → ๅ†ๆฑบๅฎš่ฆไธ่ฆๆ”พๅคง。

    ๐Ÿ’ฌ ไฝ ็›ฎๅ‰็š„้ซ˜ไฝต็™ผ็“ถ้ ธๆ˜ฏๅ“ชไธ€็จฎ?
    • ๆกๆ‰‹ๅฐ–ๅณฐ:SYN backlog / accept queue overflow?
    • ็Ÿญ้€ฃ็ทš:TIME_WAIT ๅคชๅคš、ephemeral ports ไธๅค ?
    • ๅžๅ:socket buffer、retrans、p99 latency ๆŠ–ๅ‹•?

    ไฝ ๅฏไปฅๅœจ็•™่จ€่ฒผไธŠ ss -snetstat -s ็š„้—œ้ต็‰‡ๆฎต(้ฎ่”ฝ็ถฒๅŸŸ/IP ๅณๅฏ),ๆˆ‘ๆœƒ็”จ「ๅฏๅ›žๆปพ」็š„ๆ–นๅผๅนซไฝ ๆŠŠ่ชฟๆ ก่ทฏ็ทšๆ”ถๆ–‚ๅˆฐๆœ€ๅฐ‘ๆ”นๅ‹•、ๆœ€ๅคงๆ”ถ็›Š。

    ๅปถไผธ้–ฑ่ฎ€

    ๐Ÿ”— ๅˆ†ไบซ้€™็ฏ‡ LINE Facebook X

    ๆฒ’ๆœ‰็•™่จ€:

    ๅผต่ฒผ็•™่จ€

    ๅญ—็ดš