← Devlog

ESX admin exploit: one write to users.group, and what escrow hid from us

A write to ESX's users.group made two players admin, and escrow hid who sent it. How it works and how to protect your server.

On our ESX server, two player rows in the users table changed their group from user to admin. For the next four to five hours, someone used admin commands on the players who were connected: money, items, the kind of thing that wrecks an economy in an afternoon.

We stopped it, rolled everything back and closed the hole. What we never found was which resource sent the query. That is the part this post is about.

What we could prove

The database log showed the write clearly: an UPDATE on users that set group to admin, sent through oxmysql by a resource running on our own server. Not a leaked database password, not someone on the machine. One of our resources sent it.

The obvious next step is to search the code for that query. We did, and it was not in any file we could read.

Why escrow hid the source

Dozens of resources on the server were escrowed, and several of them had database access from code we could not read. Some of them did not even build their own queries: an editable file received raw SQL text from the encrypted core and ran it. From the outside there is no way to know who builds that text or from what input.

What we found instead: SQL injection and heavy queries

While we looked, we found real problems in the parts we could read:

  • SQL injection risks in a popular paid script: column names and job filters were pasted straight into queries instead of being passed as parameters.
  • Very heavy queries in scripts nobody had profiled, running on every player action.
  • Settings that should have been in the config and were locked inside the encrypted files.

We fixed what the editable files allowed. None of it could be proven to be the source of the attack. That is what escrow means for a server owner: when something goes wrong, you cannot audit the code that runs on your own machine.

The hole was in open code

The query alone should not have made anyone admin. It did because of how ESX loads permissions. When a player joins, es_extended reads their row from users, including group, and adds the player to an ACE principal with that name:

-- es_extended, simplified: the group comes straight from the database
ExecuteCommand(('add_principal identifier.%s group.%s'):format(license, group))

From then on, the player has every permission your server.cfg grants to group.admin, and ESX’s own admin commands for the groups in Config.AdminGroups. Any resource that can write that one column can create an admin.

That makes users.group as sensitive as your admin password, and most ESX servers do not treat it that way. There are two ways to protect it, and they work best together:

  • Make sure nothing else can write it. Only your own admin tools should ever change a player’s group. No other resource should build a query that can reach that column.
  • Or stop it from being the only key. Require admins to also be listed by identifier in your server.cfg, so a changed column on its own does not make anyone admin. That needs a small change in how your ESX copy assigns the principal, so test it before rolling it out.

QBCore and Qbox store permissions differently, but the question is the same on any framework: find where admin rights come from, and check who can write there.

You cannot audit every escrowed script you buy. You can make sure that the one value standing between a player and admin is guarded, and that you hear about it the moment it changes.

How we recovered

We had full automatic database backups, kept on the server and on a separate machine. We restored the one from a few minutes before the first escalation. Every player lost the four to five hours since then, and that hurt. But the money and items that were handed out were gone, and the economy was intact.

It was not the first time backups saved the server. Years earlier, a fire at our hosting provider’s datacenter destroyed the machine and nothing on it could be recovered. With the code in Git and the database dumps on another machine, we were back in about 24 hours instead of starting from zero.

The other door: your admin panel

A web panel that reads and writes the same database is the same risk with a login page in front. If one leaked password is enough to open it, it is enough to change any player’s group. Treat it like the server: two-step verification, login rate limits and a database user with only the permissions the panel needs.

If you use Game Management, version 2.3 adds optional two-step verification, so a leaked password is not enough to open the panel, and every staff action goes to its audit log.

Checklist: protect your ESX server from admin exploits

  1. List every escrowed resource that touches the database. Those are the ones you cannot audit. Ask the author what they do with player input, and prefer open code for anything that handles money, permissions or SQL.

  2. Search your open resources for SQL built with .. or string formatting. Every value from a player should go in as a parameter, never pasted into the query:

    -- Unsafe: the player's input becomes part of the query
    MySQL.query('SELECT * FROM owned_vehicles WHERE plate = "' .. plate .. '"')
    
    -- Safe: the value goes in as a parameter
    MySQL.query('SELECT * FROM owned_vehicles WHERE plate = ?', { plate })
  3. Guard the column that makes someone admin. On ESX that is users.group. Nothing but your admin tools should be able to write it, and if you can, require admins to be listed in your server.cfg as well, so a database write alone cannot create an admin.

  4. Back up the database automatically and often, off the machine. Then restore one on a test server. A backup you have never restored is a hope, not a backup.

  5. Log admin actions and changes to player groups. You want to see the first escalation, not the fifth hour.

  6. Protect your admin panel like the server. Two-step verification, login rate limits, and a database user with only the permissions the panel needs.

How XeX scripts handle this

Every XeX script that writes to the database passes values as parameters, and everything that gives money, items or permissions is decided on the server. My paid scripts use escrow too, because it is how Cfx.re protects paid code. That is why the bridge and the config ship unencrypted, so you can read how each script talks to your framework, and why none of them writes to the users table.

Want to know what your own resources would let a player do? The Performance & Security Audit traces every place your server creates money, items and permissions, and gives you a report ranked by impact, with the fix for each item.

Related: FiveM performance: finding what lags, from resmon to the server profiler.