file attributes

[Barry R Cisna]

Hello All,

One of our older ftp servers centos 5 got hit with the shv4 rootkit,,,as
I had left ssh running mistakenly for a couple days.
Long story short I simply can not delete the two main dirs that are
created by the rootkit. Those being:
lib/libsh and /usr/lib/libsh.so.

I know the immutable bit has not been set on these dirs or the files
within. I did do an chattr -i /dir/files on the dirs just to make sure
as well. Even changing file perms to root-root the dirs and files within
can not be deleted.

I noticed when trying to rm /lib/libsh/filexyz it always comes back with
"Operation not permitted". I also notice at the end of each file name
there is the ' character. Does anyone have any idea what the ' character
suggests?

I know,I should simply reformat the box with something newer but I am
just trying to figure out firstly why the files are un-deletable.
I am going to plop in a deft live cd and see if I can delete the files
this way. Haven't had a chance to try this yet.

Thanks,
Barry Cisna

[Rob Owens]

Post by Barry R Cisna
Hello All,
One of our older ftp servers centos 5 got hit with the shv4 rootkit,,,as
I had left ssh running mistakenly for a couple days.
Long story short I simply can not delete the two main dirs that are
lib/libsh and /usr/lib/libsh.so.
I know the immutable bit has not been set on these dirs or the files
within. I did do an chattr -i /dir/files on the dirs just to make sure
as well. Even changing file perms to root-root the dirs and files within
can not be deleted.
I noticed when trying to rm /lib/libsh/filexyz it always comes back with
"Operation not permitted". I also notice at the end of each file name
there is the ' character. Does anyone have any idea what the ' character
suggests?
I know,I should simply reformat the box with something newer but I am
just trying to figure out firstly why the files are un-deletable.
I am going to plop in a deft live cd and see if I can delete the files
this way. Haven't had a chance to try this yet.

I don't know anything about that particular rootkit. But perhaps it
has provided a modified 'rm' command which refuses to delete the files
it relies on. That could be verified by using a live cd.

When trying to delete files with funny characters, it is often easier to
do using a GUI file manager. Then you just have to click on it, instead
of trying to figure out how to type it properly on the command line.

-Rob

[Jim Kinney]

Use rpm -Va to see all packages compared to their md5 checksum from rpmdb.
Replace every package that has binaries that don't match what rpm says they
should be. Use yum replace foo or rpm -F foo.
If you still can't modify that directory try a reboot to clean up the mount
tables. Start it in single user mode.


rpm -Va | egrep '^..5' | awk '{print $NF}' | xargs rpm -q --whatprovides |
sort -u

will provide the package list you need. If you are in runlevel 3 just run:

yum reinstall $(rpm -Va | egrep '^..5' | awk '{print $NF}' | xargs rpm -q
--whatprovides | sort -u)

and that will reinstall the bad packages.

Post by Barry R Cisna
Hello All,
One of our older ftp servers centos 5 got hit with the shv4 rootkit,,,as
I had left ssh running mistakenly for a couple days.
Long story short I simply can not delete the two main dirs that are
lib/libsh and /usr/lib/libsh.so.
I know the immutable bit has not been set on these dirs or the files
within. I did do an chattr -i /dir/files on the dirs just to make sure
as well. Even changing file perms to root-root the dirs and files within
can not be deleted.
I noticed when trying to rm /lib/libsh/filexyz it always comes back with
"Operation not permitted". I also notice at the end of each file name
there is the ' character. Does anyone have any idea what the ' character
suggests?
I know,I should simply reformat the box with something newer but I am
just trying to figure out firstly why the files are un-deletable.
I am going to plop in a deft live cd and see if I can delete the files
this way. Haven't had a chance to try this yet.
Thanks,
Barry Cisna
_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

[Steve]

might try to remove by inode

ls -il {file-name} to find inode of file
find . -inum [inode-number] -exec rm -i {} \; to remove

Post by Barry R Cisna
Hello All,
One of our older ftp servers centos 5 got hit with the shv4 rootkit,,,as
I had left ssh running mistakenly for a couple days.
Long story short I simply can not delete the two main dirs that are
lib/libsh and /usr/lib/libsh.so.
I know the immutable bit has not been set on these dirs or the files
within. I did do an chattr -i /dir/files on the dirs just to make sure
as well. Even changing file perms to root-root the dirs and files within
can not be deleted.
I noticed when trying to rm /lib/libsh/filexyz it always comes back with
"Operation not permitted". I also notice at the end of each file name
there is the ' character. Does anyone have any idea what the ' character
suggests?
I know,I should simply reformat the box with something newer but I am
just trying to figure out firstly why the files are un-deletable.
I am going to plop in a deft live cd and see if I can delete the files
this way. Haven't had a chance to try this yet.
Thanks,
Barry Cisna
_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

[Barry R Cisna]

Hello All,

Thanks to all that made suggestions on how to clear out the two main
rootkit directories.
I did try using a couple live cd's ,both Centos 6 and a deft CD( which I
have found usually mounts LVM disks R/W easily). Unfortunately I got the
same 'permission denied" when trying to delete the files using this
method?

I also tried the method of deleting the file by the inode number.
Strangely enough,,I get no errors,and also I never get asked to confirm
to actually delete the file. The command completes,,but the file(s) are
not deleted. Nothing shows up in system log either FYI.This method did
not work either.

One thing I did find by accident when I try and create a blank file
regardless of method,I always get error" bad file descriptor" when
actually saving the blank file into either one of the two rootkit dirs.
I am not sure what this represents. Almost as though these dirs are
mounted in some fashion although,I see nothing in mtab or fstab?
This is very strange to say the least.

It is almost as though the immutable bit is set on these files,and
dirs,,when in fact this is not the case.
Any forensic experts here?....:)
Guess I'll have to learn to make SURE I shut off ssh after remoting into
the machine in the future.

Take Care,
Barry

[Jim Kinney]

have you run the tool rkhunter? It can identify many Linux root kits. That
helps a lot in setting the next steps. I expect the rm command is
compromised on the machine itself. Very strange the live CD's failed to be
useful. They are self contained and should let you mount and attrib and rm
all you like.

Note: if you turn off ssh, you can't remote back in. The trick is to keep
ssh updated (and any parts it depends on), deny root ssh connection,
requires ssh keys (very, very secure) and block password access.

Post by Barry R Cisna
Hello All,
Thanks to all that made suggestions on how to clear out the two main
rootkit directories.
I did try using a couple live cd's ,both Centos 6 and a deft CD( which I
have found usually mounts LVM disks R/W easily). Unfortunately I got the
same 'permission denied" when trying to delete the files using this
method?
I also tried the method of deleting the file by the inode number.
Strangely enough,,I get no errors,and also I never get asked to confirm
to actually delete the file. The command completes,,but the file(s) are
not deleted. Nothing shows up in system log either FYI.This method did
not work either.
One thing I did find by accident when I try and create a blank file
regardless of method,I always get error" bad file descriptor" when
actually saving the blank file into either one of the two rootkit dirs.
I am not sure what this represents. Almost as though these dirs are
mounted in some fashion although,I see nothing in mtab or fstab?
This is very strange to say the least.
It is almost as though the immutable bit is set on these files,and
dirs,,when in fact this is not the case.
Any forensic experts here?....:)
Guess I'll have to learn to make SURE I shut off ssh after remoting into
the machine in the future.
Take Care,
Barry
_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

--
--
James P. Kinney III

As long as the general population is passive, apathetic, diverted to
consumerism or hatred of the vulnerable, then the powerful can do as they
please, and those who survive will be left to contemplate the outcome.
- *2011 Noam Chomsky

http://heretothereideas.blogspot.com/
*

[Les Mikesell]

Post by Barry R Cisna
It is almost as though the immutable bit is set on these files,and
dirs,,when in fact this is not the case.

Did you check this or try to clear it with the lasttr/chattr tools
from the livecd boot?

[Barry R Cisna]

Hello,

thanks for the additional suggestions.
Yes. I have run rkhunter,,and rkhunter does in fact show that this
machine has hanged with shv4.

Also i did boot up one more time with the deft live cd to double check
the lsattr of each of the two directories,,and neither dir shows the
immutable bit is set on the dir,,or any of the files that will not
delete?

Also for completeness I did not mention previously I did copy known good
binaries from a good server to replace the (about a dozen binaries) that
was replaced by shv4 kit. I could tell fairly easily as the replaced
binaries had a user of 114 and group of 112.

The machine does not really show any odd processes,,or netstat stuff
running,,,but Id simply like to figure out why these files will not
delete.


Thanks again,
Barry

[Jan Middelkoop]

Everybody loves a good mystery. :-)

Is there anything in /var/log/messages, or in dmesg when you try to
delete these files?

Kindest regards,

Jan Middelkoop
Recreatie en Zorg Groep B.V.
--
Website: http://www.recreatie-zorg.nl/
E-mail: ***@recreatie-zorg.nl
Telefoon: [phone]

Post by Barry R Cisna
Hello,
thanks for the additional suggestions.
Yes. I have run rkhunter,,and rkhunter does in fact show that this
machine has hanged with shv4.
Also i did boot up one more time with the deft live cd to double check
the lsattr of each of the two directories,,and neither dir shows the
immutable bit is set on the dir,,or any of the files that will not
delete?
Also for completeness I did not mention previously I did copy known good
binaries from a good server to replace the (about a dozen binaries) that
was replaced by shv4 kit. I could tell fairly easily as the replaced
binaries had a user of 114 and group of 112.
The machine does not really show any odd processes,,or netstat stuff
running,,,but Id simply like to figure out why these files will not
delete.
Thanks again,
Barry
_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see<http://www.k12os.org>

[Jim Kinney]

try a dd if=/dev/zero of=/<file path and name> bs=1024 count=1024

this _should_ overwrite the file(s) for 1M with 0s.

Also check selinux status with ls -laZ and see if selinux got turned on or
changed 'getenforce' to show, 'setenforce [01]' to temp change.

Post by Barry R Cisna
Hello,
thanks for the additional suggestions.
Yes. I have run rkhunter,,and rkhunter does in fact show that this
machine has hanged with shv4.
Also i did boot up one more time with the deft live cd to double check
the lsattr of each of the two directories,,and neither dir shows the
immutable bit is set on the dir,,or any of the files that will not
delete?
Also for completeness I did not mention previously I did copy known good
binaries from a good server to replace the (about a dozen binaries) that
was replaced by shv4 kit. I could tell fairly easily as the replaced
binaries had a user of 114 and group of 112.
The machine does not really show any odd processes,,or netstat stuff
running,,,but Id simply like to figure out why these files will not
delete.
Thanks again,
Barry
_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

--
--
James P. Kinney III

As long as the general population is passive, apathetic, diverted to
consumerism or hatred of the vulnerable, then the powerful can do as they
please, and those who survive will be left to contemplate the outcome.
- *2011 Noam Chomsky

http://heretothereideas.blogspot.com/
*

[Josh Malone]

Post by Jim Kinney
try a dd if=/dev/zero of=/<file path and name> bs=1024 count=1024
this _should_ overwrite the file(s) for 1M with 0s.
Also check selinux status with ls -laZ and see if selinux got turned on or
changed 'getenforce' to show, 'setenforce [01]' to temp change.

Or just do what you really should do when a system gets rooted:
re-install it.

Personally, I'd be most concerned with finding the initial attack
vector. Every time we've dealt with a system intrusion before, we always
discovered how they got in in the first place. Usually it's compromised
account credentials (read "passwords") that came from either other
systems or a successful phishing campaign. Turning off SSH should not be
your ultimate solution for network security.

Did the attackers brute-force your root password? Or did they find some
remote vulnerability?

If you don't know, to me that's even more reason to re-install the
system, possibly with a newer release. I've been very happy with my LTSP
install on RHEL6 and that's rather easy to keep up to date.

-Josh

[Barry Cisna]

Hello All,

I did try the dd if,,, of as suggested in a post to change file
size,,etc. After doing this routine i still get 'permission denied,,when
trying to delete each file after the convert.
SELinux is still disabled as before.
Nothing is ever shown in any logs ,either system or secure,,,when trying
to rm a file.

It seems the key in this is, when I try and create a blank text file and
save to either of the libsh(rootkit) dirs I get 'bad file descriptor',
thrown.
very odd?

Not that it makes any diff,,but of course in searching the logs I did
find the ip address that dropped in this rootkit was from China,
surprise,,surprise,,,.


Thanks again,
Barry

[Jim Kinney]

I missed the part about these were dirs and not files.
The dir names may have nonprinting characters so your work to change things
is failing on wrong name.
At this point you need to replace the drive or it's contents. Unless you
can work at the inode level to wipe very specific bits, the level of effort
exceeds realistic time frames. As long as those dirs exist, you're running
a compromised system.

Post by Barry R Cisna
Hello All,
I did try the dd if,,, of as suggested in a post to change file
size,,etc. After doing this routine i still get 'permission denied,,when
trying to delete each file after the convert.
SELinux is still disabled as before.
Nothing is ever shown in any logs ,either system or secure,,,when trying
to rm a file.
It seems the key in this is, when I try and create a blank text file and
save to either of the libsh(rootkit) dirs I get 'bad file descriptor',
thrown.
very odd?
Not that it makes any diff,,but of course in searching the logs I did
find the ip address that dropped in this rootkit was from China,
surprise,,surprise,,,.
Thanks again,
Barry
_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

[b***@thealmquists.net]

You may be able to use tab completion to get the correct file or folder name.

Post by Jim Kinney
I missed the part about these were dirs and not files.
The dir names may have nonprinting characters so your work to change things
is failing on wrong name.
At this point you need to replace the drive or it's contents. Unless you
can work at the inode level to wipe very specific bits, the level of effort
exceeds realistic time frames. As long as those dirs exist, you're running
a compromised system.

Post by Barry R Cisna
Hello All,
I did try the dd if,,, of as suggested in a post to change file
size,,etc. After doing this routine i still get 'permission denied,,when
trying to delete each file after the convert.
SELinux is still disabled as before.
Nothing is ever shown in any logs ,either system or secure,,,when trying
to rm a file.
It seems the key in this is, when I try and create a blank text file and
save to either of the libsh(rootkit) dirs I get 'bad file descriptor',
thrown.
very odd?
Not that it makes any diff,,but of course in searching the logs I did
find the ip address that dropped in this rootkit was from China,
surprise,,surprise,,,.
Thanks again,
Barry
_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

[Jim Kinney]

for dirname in $(ls -1 <path to bad dir>); do echo ">>${dirname}<<"; done
That will put the >><< around the name. A simple space will cause mayhem.

Post by b***@thealmquists.net
You may be able to use tab completion to get the correct file or folder name.

Post by Jim Kinney
I missed the part about these were dirs and not files.
The dir names may have nonprinting characters so your work to change

things

Post by Jim Kinney
is failing on wrong name.
At this point you need to replace the drive or it's contents. Unless you
can work at the inode level to wipe very specific bits, the level of

effort

Post by Jim Kinney
exceeds realistic time frames. As long as those dirs exist, you're running
a compromised system.

Post by Barry R Cisna
Hello All,
I did try the dd if,,, of as suggested in a post to change file
size,,etc. After doing this routine i still get 'permission denied,,when
trying to delete each file after the convert.
SELinux is still disabled as before.
Nothing is ever shown in any logs ,either system or secure,,,when trying
to rm a file.
It seems the key in this is, when I try and create a blank text file and
save to either of the libsh(rootkit) dirs I get 'bad file descriptor',
thrown.
very odd?
Not that it makes any diff,,but of course in searching the logs I did
find the ip address that dropped in this rootkit was from China,
surprise,,surprise,,,.
Thanks again,
Barry
_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

--
--
James P. Kinney III

As long as the general population is passive, apathetic, diverted to
consumerism or hatred of the vulnerable, then the powerful can do as they
please, and those who survive will be left to contemplate the outcome.
- *2011 Noam Chomsky

http://heretothereideas.blogspot.com/
*

[lpadumorvan]

Hi

I want to play http://www.example.com/video.mpg

Firefox and vlc are installed on the server and on the client

I use centos6_64 for the server and centos6_32 for the client.

With firefox in localapps mode and vlc in server mode
The video.mpg is downloading in the /tmp of the server and vlc play the file
with success.(the window "open with vlc multimedia player" opens correctly)

But with firefox and vlc in localapps mode
The video.mpg is downloading in the /tmp of the server and vlc can't play
the file. (the window "open with vlc multimedia player" opens correctly)
The file video.mpg is present in /tmp on the server.

The error on VLC is:
"File reading failed:

VLC could not open the file "/tmp/video.mpg".

Your input can't be opened:

VLC is unable to open the MRL 'file:///tmp/video.mpg'. Check the log for
details"

Thank for your help

Bests regards

VUILLET Damien

[Jim Kinney]

Ah! I'll bet vlc is looking on the CLIENT /tmp and not the SERVER /tmp.

You can set up the server /tmp to be mounted by the clients. I've not
thought through what the performance/security issues will be. I don't see
performance as much of a problem as security is. Things _should_ be
UID-safe as long as all shared-space users originate their security access
(password, master user list on AD or LDAP) from the server.

Post by lpadumorvan
**
Hi
I want to play http://www.example.com/video.mpg
Firefox and vlc are installed on the server and on the client
I use centos6_64 for the server and centos6_32 for the client.
With firefox in localapps mode and vlc in server mode
The video.mpg is downloading in the /tmp of the server and vlc play the file
with success.(the window "open with vlc multimedia player" opens correctly)
But with firefox and vlc in localapps mode
The video.mpg is downloading in the /tmp of the server and vlc can't play
the file. (the window "open with vlc multimedia player" opens correctly)
The file video.mpg is present in /tmp on the server.
VLC could not open the file "/tmp/video.mpg".
VLC is unable to open the MRL 'file:///tmp/video.mpg'. Check the log for
details"
Thank for your help
Bests regards
VUILLET Damien
_______________________________________________
K12OSN mailing list
https://www.redhat.com/mailman/listinfo/k12osn
For more info see <http://www.k12os.org>

--
--
James P. Kinney III

As long as the general population is passive, apathetic, diverted to
consumerism or hatred of the vulnerable, then the powerful can do as they
please, and those who survive will be left to contemplate the outcome.
- *2011 Noam Chomsky

http://heretothereideas.blogspot.com/
*

[Burke Almquist]

Post by lpadumorvan
With firefox in localapps mode and vlc in server mode

Why not simply put VLC in localapps mode as well?