Jump to content

HELP! Is it Possible to Recover Found Files in Place? (And More)


FLskydiver

Recommended Posts

Also ... Is Recuva unable to recover files over 4 GB?

___

 

Windows 7 64bit | Recuva Version 1.44.778 for Win 64bit

 

How's this for ironic --- while attempting to back up a massive amount of my date; I just lost an even larger amount.

 

I'm sorry this got a little long, but please read through. Thanks!

 

I recently aquired a Western Digital My Passport for Mac, 2 TB External, Portable (2.5") USB Hard Drive (that I formatted NTFS) and I used this to backup 161 GB of my most sensitive data and have since been using it to store newly created video content in what was its vast amount of remaining free space. In the last two months, I'd just about filled the thing -- it only had about 90 GB of free space remaining this morning.

 

Everything I added to the Passport Drive was finished data I had no need to mess with further (apart from viewing it) -- so fragmentation should be nearly non-existent on this drive if Windows was in any way sensible about the way it wrote down the data. Apart from ocassionally renaming a few of the files, or "cutting and pasting" them to a different folder on the same drive (which I'm almost certain doesn't move the data at all) -- I haven't messed with any of the files once I've dropped them on the passport.

 

I did however consistently delete the originals from the internal hard disk where they were first created and collected for space saving reasons. Until today, there were no backups of the video content (over 1.7 TB's worth) on that Passport.

 

So today I've been digging up all the old drives I can find and trying to copy the video files from the Passport over onto the old drives. None of them are large enough to hold all the content, however, and some had other data I couldn't delete, so I've had to backup the video data in chunks.

___

 

Well, tonight after three long attempts (my computer kept freezing) I finally succeeded in backing up the first 1,100 GB or so of the video content to one of the old drives when diaster struck.

 

Here's how it went down:

 

My computer had been left unattended while the copy was taking place -- copying from one external hard drive (the My Passport 2TB) to another external hard drive (a Seagate 1.5 TB 3.5" internal desktop drive in an external enclosure) with both connected to my 6 year old Dell Inspiron 1720 laptop via US 2.0 (the Ispiron lacks the USB 3.0 capability of the My Passport) takes a very very long time.

 

When I checked to find the process had finally completed, everything looked excellent. I'd just about filled the 1.5 TB external drive; and only had 600 GB or so of files left to find space on other hard drives for (if I could find it at all!).

 

One thing I do after copying data is check to make sure the folder properties of the source and destination directories match up -- same number of files, same number of folders, same total number of bytes. I'm pretty sure that went well.

 

I'm a little fuzzy on what happened next.

 

At some point; it dawned on me that I can no longer right-click and see the option to "select all" in the folder properties menu anymore in Windows 7 (that I have been using nor for, I don't know, 2 or 3 years?) and for some reason this really bothered me --- I've got CDO. It's like OCD, except the letters are in alphabetical order, like they ougt to be. So anyways .... I got momentarily distracted and did a quick google search and found a simple little regedit file that I could add to the registry to restore the feature. I added it, and it worked. No big deal.

 

However, when I next navigated to the folder on the Passport drive where I'd been copying the data from (which I did immediately), the entire directory was blank!!! There was nothing at all in the folder!! 528 folders -- gone!

 

The passport drive now shows "1.10 TB Free Space of 1.81 TB" where it ought to show only about 90 GB (i.e 0.09 TB) free space, Over 1 TB of data just disappeared in an instant without even a puff of smoke!

 

PANIC! :o

 

I haven't a clue what I did. Maybe I somehow accidentally hit both the "Control-A" hotkey on my keyboard (thanks, AutoHotkey!) and then "Delete" while focused on that directory, but I can't imagine how, because I was too lazy to get up and get the keyboard in the first place and that's why I went looking for the "Selct-All in Right-Click" fix. Maybe adding the registry keys while the entire directory was selected somehow nuked them? I just don't know.

 

But!! Luckily, I'd had the forethought to install RECUVA! ;)

 

So I ran a scan with that.

 

But at somepoint I also opened the recycle bin (that I always empty incessantly -- my OCD again -- I even have a handly little AutoHotkey mod so I only need to hit the "Insert Key" to do it) and just 42 of my 528 missing folders were in there. Just 92 GB! That's only 9% of my missing data! I don't know why they wound up in the recycle bin and none of the others didn't; but that's that.

 

And want to know the worst part? All 42 of those folders were ones I had succesfully backed up. Not any were of the 193 I hadn't yet. Grrr!

 

***But hey, wait!! That's also just about how much free space my drive had before my data got nuked! Does that mean something??***

 

QUESTIONS:

 

When Recuva did its search, it did seem to find the vast majority of the files in the 528 missing folders.... but ALL fifty files larger than 4 GB (4,200,000 KB) were tagged RED and marked as "Unrecoverable | File's data could not be found on the disk." even though I haven't written anything to the disk at all since before the files went missing. The rest are green.

 

Is Recuva truly unable to recover files larger than 4 GB?

 

and

 

If I attempt to recover 528 folders totallying something like 1 TB of data in place (I don't have anywhere else to recover the data) will Recuva simply fix the MFT record (or whatever) and make the files visible again, or attempt to rewrite the data elsewhere on the disc in "free space" and thus destroy everything it is trying to save? I couldn't tell exactly from the documentation.

 

If Recuva can't restore in place, like I think maybe the Recycle Bin does, is there any other tool that can?

 

Thanks for your time and assistance!

Link to comment
Share on other sites

  • Moderators

I'm not sure that I've grasped the full story, if you've backed up all but 600 gb from the Passport then you'll be looking for less than 1.1 tb?

 

I have found, from experiment and from comments elsewhere, that NTFS manages deletion of files larger than 4 gb differently from those smaller than 4 gb. In the smaller files the entry in the MFT is (relatively) simply flagged as deleted, and the dataruns - the fields that hold the address and number of clusters for each extent - are untouched. This is how Recuva can find and recover the file data. In files larger than 4 gb the dataruns are overwritten by NTFS. Although the file size is still available, the addresses of the data are not, and Recuva will show the Data not Found on Disk message. In this case no recovery programme can recover data from information in the MFT.

 

This also happens in smaller files which are in many extents. If the list of extents is too long to fit into a single MFT record it is placed in a MFT extension record. These too are erased on file deletion, leaving the file data unrecoverable.

 

You can run a deep scan. This will search all the clusters on the disk looking for a recognisable file signature, and quite likely will find some of your lost files, even if it will not be able to find a file name. However this will only find the first extent of a file, as there is no information linking subsequent extents to the first extent, and subsequent extents do not have file signatures. It's pot luck, just because you have a lot of space doesn't mean that your data will be in single extents. Possibly data recovery specialists can find these extents, but not me.

 

Recuva, and other file recovery programmes, can't recover in place. The concept is complex, and NTFS will not allow any application to modify the MFT. You will have to recover to free space elsewhere, as the risk of overwriting data on the Passport is too high. The Recycler manages this because it doesn't delete the files at all, it renames then and puts them in a non-visible system folder, and just renames them back again on restore.

 

I don't think a Scan for Non-Deleted Files will show much, as it doesn't appear as if you've formatted the Passport. But it's fast and non-destructive so might be worth a try before a deep scan, which will take a long time. Apart from that, maybe a quote from a data recovery specialist?

Link to comment
Share on other sites

Pros have the necessary tools to rebuild those files. And that is my recommendation.

 

Rhetorical Questions follow:

WHY is your system locking up in the first place? Performing disk operations of any sort on an unstable system is a no-no!

 

The Recycle Bin has a setting to retain X-amount of data, typically defaults to 10%, but you can manually change it. When it starts to fill up, (10% of 1TB is about 85 GB), it pushes out the older stuff. A rolling buffer.

 

For files bigger than 4gb, and/or fragmented, a specialist will actually write (or has in his toolbox) software that will scan the entire disk and fit together fragments that most likely to belong to each other. And if the software doesn't match the fragments it presents the option to do it manually. There is software that is coming available to the general public that does this, but it isn't cheap either. This is being popularized with JPG repair kits.

 

A specialist will also clone the disk (seeing that this is a logical failure) and work from there.

 

There really are no programs that recover directly to the source without risking damaging other files. You CAN do it. But the program will go through normal Windows channels, which means the recovered data will be placed elsewhere - risking overwriting the next to-be-recovered file. This idea works for small amounts of small files, and you're still playing a game of statistics.

Link to comment
Share on other sites

  • 2 weeks later...

Thank you both for your replies. I apologize that I've taken this long to respond.

I'd like to answer your comments and questions in-line below and then update you all with my findings and comments of my own.

 

I'm not sure that I've grasped the full story, if you've backed up all but 600 gb from the Passport then you'll be looking for less than 1.1 tb?

 

I deleted about 1.20 TB (I know this now) from the Passport drive; but after I began backing it up, and thus had only half (600 GB) of the deleted 1.2 TB of data elsewhere.

 

I have found, from experiment and from comments elsewhere, that NTFS manages deletion of files larger than 4 gb differently from those smaller than 4 gb. In the smaller files the entry in the MFT is (relatively) simply flagged as deleted, and the dataruns - the fields that hold the address and number of clusters for each extent - are untouched. This is how Recuva can find and recover the file data. In files larger than 4 gb the dataruns are overwritten by NTFS. Although the file size is still available, the addresses of the data are not, and Recuva will show the Data not Found on Disk message. In this case no recovery programme can recover data from information in the MFT.

 

This also happens in smaller files which are in many extents. If the list of extents is too long to fit into a single MFT record it is placed in a MFT extension record. These too are erased on file deletion, leaving the file data unrecoverable.

 

You can run a deep scan. This will search all the clusters on the disk looking for a recognisable file signature, and quite likely will find some of your lost files, even if it will not be able to find a file name. However this will only find the first extent of a file, as there is no information linking subsequent extents to the first extent, and subsequent extents do not have file signatures. It's pot luck, just because you have a lot of space doesn't mean that your data will be in single extents. Possibly data recovery specialists can find these extents, but not me.

 

Recuva, and other file recovery programmes, can't recover in place. The concept is complex, and NTFS will not allow any application to modify the MFT. You will have to recover to free space elsewhere, as the risk of overwriting data on the Passport is too high. The Recycler manages this because it doesn't delete the files at all, it renames then and puts them in a non-visible system folder, and just renames them back again on restore.

 

I don't think a Scan for Non-Deleted Files will show much, as it doesn't appear as if you've formatted the Passport. But it's fast and non-destructive so might be worth a try before a deep scan, which will take a long time. Apart from that, maybe a quote from a data recovery specialist?

Pros have the necessary tools to rebuild those files. And that is my recommendation.

 

Rhetorical Questions follow:

WHY is your system locking up in the first place? Performing disk operations of any sort on an unstable system is a no-no!

 

I'll give you an answer anyway: I have no other choice. Can't afford anything new.

 

I have absolutely no idea why my laptop keeps freezing, other than, it is getting old. Its an Inspiron 1720 with a Core2 Duo T7500 processor @ 2.2 GHz, 4 GB of Ram, and NVIDIA GeForce 8600M GT graphics. Purchased in the second half of 2007. I've ruled out software issued for the following reasons: A clean install of Windows 7 64bit on a new hard disk didn't help; and the computer often takes a while to load the BIOS (where originally it only took a few seconds). Sometimes it freezes before WIndows even loads.Sometimes it freezes hours later with no system overhead or computational load whatsoever. Most recently it froze just as I'd begun writing this reply (though I was backing up data in the background) after running fine for almost 20 hours. Often when I reboot it after a crash, it loads the BIOS much faster an without issue. Perhaps it doesn't like the cold. I've been keeping my cabin in the 40-45° F range most of the winter -- again, to save my limited resouces, I only need keep the place warm enough to keep the interior pipes from freezing (external pipes are electrically heated) -- you'd be surprised how easy it is to get used to. In any case, I seem to have some bad hardware, but all diagnostics I've run check out, so I can't pin it down.

 

When it does lock up, it does so instantly, without warning, seemingly randomly. There is no detectable slow down or anything, it just stops. No response to any keyboard imput, or anything else but a hard power-off. I have the laptop plugged into my TV as a monitor -- when it freezes the picture remains. Opening the lid does not switch the picture back to the TV. Only a hard power down brings it back. I wish I could figure it out, but I have been unsuccessful.

 

The Recycle Bin has a setting to retain X-amount of data, typically defaults to 10%, but you can manually change it. When it starts to fill up, (10% of 1TB is about 85 GB), it pushes out the older stuff. A rolling buffer.

 

THIS! It pains me how easy it would have been to avoid this catastrophe. I've since set the maximum size of all my recycle bins (on a per drive basis) to 100% of the size of the disk (just enter 9,999,999 MB and it will default to 100%). Why Windows does not do this by default, especially when you delete things all at once that exceed it's arbitrary, artificially reduced capacity, I will never know. Had I done this prior, all 1.2 TB of my data would have simply been put in the recycle bin. There is no 4GB cap on files being recoverable from the recycle bin, because as mentioned above, Windows simply renames the files.

 

For files bigger than 4gb, and/or fragmented, a specialist will actually write (or has in his toolbox) software that will scan the entire disk and fit together fragments that most likely to belong to each other. And if the software doesn't match the fragments it presents the option to do it manually. There is software that is coming available to the general public that does this, but it isn't cheap either. This is being popularized with JPG repair kits.

 

A specialist will also clone the disk (seeing that this is a logical failure) and work from there.

 

There really are no programs that recover directly to the source without risking damaging other files. You CAN do it. But the program will go through normal Windows channels, which means the recovered data will be placed elsewhere - risking overwriting the next to-be-recovered file. This idea works for small amounts of small files, and you're still playing a game of statistics.

 

Ok: So first, let me say thanks that a program as awesome as Recuva exists and is even FREE - that's just excellent - thanks to all involved.

 

But it does have the limitation (as nearly every other un-deleter out there does) of leaving useres SOL on files they've lost which are greater than 4GB in size.

 

I found one that does not have this limitation: As mentioned, it isn't free, and it isn't really what I'd call cheap either. But it works, and it has saved my ass. It's called R-Studio, and apparently (from what I read in other forums) versions 5.4 and above can help with files over 4 GB (I used 6.1). Unfortunately, it too, can't recover in place. That's is soooo frustrating. The data's right there! It didn't go anywhere? Why can't we just add it back to the index?

 

I really want to take the WIndows Team that was responsible for making this seemingly simple and forseeable issue so nearly impossible and complex to overcome and shake some sense into them. For starters, how about making sure you get a confirmation before deleting items that won't fit in the recycle bin? Or before deleting 1 TB + of data??

 

A funny thing though about R-Studio. Even though it can find the big files which bypassed the recycle bin and recover them, it can't recover their folder path (well, their original folder path -- it comes up with the random string of characters from the recycle bin it bypassed). So for files under 4GB, Recuva still works the best. Plus, it scans a heck of a lot faster. When I ran the R-Studio disc scan on my 2TB passport, it was set to take about 20 hours. Whichmore or less equals how long it takes to read and or write 2TB of data to it all at once.

 

Funny thing is, I paniced about that because asking my computer to work hard for 20+ hours in a row is pretty schetchy thing to ask these days. And as feared, my computer froze about 10 hours in. :-(

 

Thank goodness, however, that R-Studio seems to save its findings to disc as it progresses, and wasn't blown up by the sudden lock up and improper shut down. Even more surprisingly, it was able to locate and completely recover ALL my data even though it had only scanned less than half the source disc. Makes me wonder how necessary the scan was, afterall. Plus, R-Studio makes reovery really easy by giving you lots and lots of filters to use ... based on file extensions sorted by file type, time, size, or anything else you can think of. Very much a help, because even though I needed to reover mountains of data. I only wanted to recover the files I hadn't previously backed up elsewhere.

 

So here I am, over two weeks later, STILL copying data back to the passport (right on top of where it already was). Because I had to find somewhere else to recover the data (scavenging hard drives from even older computers in order to shuffle enough data around to have somewhere to recover the data too, and boy, it takes a massively long time to move TB's of data over nothing faster than USB 2.0. Weeks lost because I accidentally hit a couple of the wrong keys on my keyboard ... and didn't know the"last chance" recycle bin had a really big apostrophe on it. This is just insane.

 

Thanks for your time..

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.