Random ramblings on hacking, coding, fighting with infrastructure and general tech
Internetwache CTF 2016 – EquationSolver – Exploit - 60 points
Posted: 22 Feb 2016 at 18:05 by Codehead

### Challenge:

EquationSolver (exp60)

### Description:

I created a program for an unsolveable equation system. My friend somehow forced it to solve the equations. Can you tell me how he did it?

Service: 188.166.133.53:12049

## Solution:

Netcat is always my first option for this type of challenge:

```> nc 188.166.133.53 12049
Solve the following equations:
X > 1337
X * 7 + 4 = 1337
Enter the solution X:
```

Lets do the obvious answer first:

1337 – 4 = 1333 1333 / 7 = 333.25

```> nc 188.166.133.53 12049
Solve the following equations:
X > 1337
X * 7 + 4 = 1337
Enter the solution X: 333.25
You entered: 333
333 is not bigger than 1337
WRONG!!!
Go to school and learn some math!
```

OK, that was never going to work, but we can gain some useful info from the result. The mantissa is dropped from the input, so we’re dealing with integers.

Integers can overflow and wrap around, so we have a chance of supplying a a very large number which will pass the greater than 1337 test, but once multiplied, will overflow, wrap around and resolve to 1337.

I assumed a 32bit int as that is the norm in most languages I’ve used. An unsigned 32bit integer has a maximum value of 4,294,967,295, lets try starting from there:

4,294,967,295 / 7 = 613566756 (round down) 1337 – 4 = 1333 1333 / 7 = 190 (round down) 613566756 + 190 = 613566946

```> nc 188.166.133.53 12049
Solve the following equations:
X > 1337
X * 7 + 4 = 1337
Enter the solution X: 613566946
You entered: 613566946
613566946 is bigger than 1337
1330 is not equal to 1337
WRONG!!!
Go to school and learn some math!
```

One out! Probably shouldn’t have rounded down.

```> nc 188.166.133.53 12049
Solve the following equations:
X > 1337
X * 7 + 4 = 1337
Enter the solution X: 613566947
You entered: 613566947
613566947 is bigger than 1337
1337 is equal to 1337
Well done!
IW{Y4Y_0verfl0w}
``` Categories: Hacking CTF