Problems with hidden files (Archos MP3)

perhaps those are the resource forks? how do you accomplish said copy?

do your mp3's have resource forks? how can you tell in OSX?

% ls -l /pathto/song.mp3/rsrc

if it's size is zero, no rez fork. non-zero = rez fork

if there are rez forks, what are they? why would an mp3 have a rez fork? did you paste an icon onto the mp3 file?

on a know rez-less mp3, i pasted an icon into its get info window, and viola...

$ ll /.../sw.mp3/rsrc
-rwxr--r-- 1 me wheel 42k Feb 15 17:59 /.../sw.mp3/rsrc*

42K rez fork

if that's your case, let me know and i may have a snappy solution for you.

Merv,
I've been trying what you have suggested and haven't come up with much. It could be my lack of any Unix experience. This is a condensed version of my terminal window.

[localhost:/volumes] laroche% ls -l STUDIO_20/"Sacha Mix"/K-paz/20-while.mp3/rsrc
ls: STUDIO_20/Sacha Mix/K-paz/20-while.mp3/rsrc: Not a directory
[localhost:/volumes] laroche% ls -l STUDIO_20/"Sacha Mix"/K-paz/20-while.mp3
-rwxrwxrwx 1 laroche wheel 4388108 Feb 6 21:52 STUDIO_20/Sacha Mix/K-paz/20-while.mp3
[localhost:/volumes] laroche% ls -l STUDIO_20/"Sacha Mix"/K-paz/.20-while.mp3/rsrc
ls: STUDIO_20/Sacha Mix/K-paz/.20-while.mp3/rsrc: No such file or directory

I copied the files by drag-and-drop. I have added an icon for the drive, but the individual files should be icon free. They do, however, show up with a quicktime icon in the finder.

The hidden files don't show up in my finder as I thought they did; Snax is the program I am using to see them. They are 82k. When I look at them with the text editor I see the word "MPG3hook". I hope this confirms your hunch about rez forks. Thanks for the help.
-LaRoche

Do the ls -l command on the mp3's on your harddrive, not the mp3 player..

yep, rez forks. and the size may be dependent on the geomtry of your hard drive.

i did a little experiment:

---
snatched an iTunes .mp3 file to a scratch area and made sure it's rez fork was zero.

did a finder get info and changed its ( open with application ) to quicktime

that added a ~42K rez fork here (disk partition size is 7GB)

changed it back to open with iTunes

that truncated the rez fork size to 286 bytes. still, that's a long way from zero, in miles and fussy mp3 players.
---

now, how to solve your problem? well, this is a perfect problem for some command line wrangling to delete the resource forks, but i'd like to think there's a simpler solution for some users. but let me know if you're willing to go commando and perhaps we could come up with a bullet-proof script.

first off, your mp3's have a QT profile. change them all to open with iTunes via the get info window, and perhaps that will prevent them from inheriting an 82K rez fork when they are touched.

perhaps OS9 File Buddy (OSX not ready for prime time yet) ?

SNAX allows you to delete rez forks, but it's one file at a time thru its get info window.

MP3 Rage? that's what i've used extensively to 'normalize' all my mp3's, but i don't know if that 'normalize' rez forks was a side effect, or intentional.

any other file mashers out there that can do this?

Doesn't the mv command *not* copy resource forks? Perhaps he could use the mv command to transfer the files, avoiding resource forks alltogether?

tried the mv command in place to see. didn't work. note that i tried this with both OSX mv and fink's mv


% ll sw.mp3
-rwxr--r-- 1 merv wheel 3.9M Feb 16 12:17 sw.mp3*

% ll sw.mp3/rsrc
-rwxr--r-- 1 merv wheel 286 Feb 16 12:17 sw.mp3/rsrc*

% mv sw.mp3 swx.mp3

% ll swx.mp3/rsrc
-rwxr--r-- 1 merv wheel 286 Feb 16 12:17 swx.mp3/rsrc*


the mv command merely changed the name entry in the directory. even moving to a subdir took the rez fork along with it. odd that.

% mkdir foo
% mv swx.mp3 foo/
% ll foo/swx.mp3/rsrc
-rwxr--r-- 1 merv wheel 286 Feb 16 12:17 foo/swx.mp3/rsrc*

only by issuing a target on another volume, did the mv command strip the rez fork. hmmmm. i'm afraid this isn't a good solution if the user doesn't have the free disk space on another partition.
---

AKcrab, also consider that the files of interest here may be filed in a dir hierarchy that prohibits simple GUI or command line mv's, e.g.,

/volumes/somePart/music/genre/artist/album/song.mp3

I think I am getting there, but still have a ways to go. I am now certain they are Rez forks; Snax identifies the files on my MP3 player as having them (but not the files in my Itunes library). I can remove them from the files via the get info box, but it is easier to just drag all of the hidden files into the trash (I've been doing that). I have tried to change the type code and creater code to nothing using MP3 Rage, but it doesn't seem to work. Supposedly empty file types still create forks after I make the copies. It doesn't remove them from files that already have forks either. I think this is moving in the right direction though. It's funny that my files have Quicktime icons anyway since I imported them with Itunes. How did you change the file type in Itunes? I didn't see this option in the get info window.

-LaRoche

p.s. I'm willing to go guerrilla.

you don't change file types in iTunes. i think if itunes sees that the data has an mpg3 audio signature, it'll just play it and leave file type/creator alone. this is good.

MP3 Rage changing type and creator code to nothing is not a solution, but changing type and creator code to the iTunes sig 'mp3!' and 'hook' could eliminate that pesky QT signature, and possibly the 42K (82K) QT iconage rez fork.

caveat: you can change an entire disk of files to type/creator in MP3 Rage, but you had ought to know that all the files should be mp3s. in other words, i wouldn't like to point that weapon at my 'cranky letters to the editor' folder.

so, with that said, are all your mp3's that need work under one top level directory? and there are no other file types in there?

then, i would use MP3 Rage to [ change file attributes ] and select your starting folder and change [x] type code: [....] [:] select iTunes from the drop down menu here. you may want to make a little test staging area to see that you're getting your desired results.

note: that change may not propagate in the finder for a while. finder is a little (sometimes very) retarded about these changes and still uses it's cache for file views. you may need to logout to flush this cache problem.

well, that's a start, i would do that (have done that) anyways, and then we can examine those results and see. let's make small changes and examine the results.

Well, I should have heeded your advice and practiced with a few folders. All of my MP3's are treed from the same directory, so I unleased MP3 Rage on my local collection, as well as on all of the files on my MP3 jukebox. I set the filed type to Itunes MP3!, and now all of the files in my player have the 82k rez files (including my main bunch of files copied with OS 9)! I think what I need now is some sort of script to delete all of them. I have about 3,000 files organized by directory (/Volumes/Studio20/artist/album/01- track.mp3). A script that would automatically delete rez files from a given directory tree would solve the problem to my satisfaction. Any suggestions?

Thanks again, -LaRoche

LaRoche,

since i don't have your mp3 player here, i would like to run a few short tests before we write a rezZap script that doesn't solve the ultimate problem.

make a scratch directory and copy some problem mp3s in there.

then, in terminal

cd /pathto/scratch
ls -l <filename.mp3>/rsrc
rm <filename.mp3>/rsrc
ls -l <filename.mp3>/rsrc

fill in the <xxx> as appropriate for the files

does that get you zero length rez forks?

if so, then push those files up to the mp3 player and check them there.

if they're fixed, then we can script the clean up.

but, i'm concerned that they will regenerate their icon rez forks like a frogs leg. thinking... (sniff, what's burning?)

I'm starting to understand what is going on here. I copied freshly imported Itunes to a scratch directory and got zero rez files (but no hidden files). However when I tried rm "Track 01.mp3"/rsrc , I deleted the mp3 file as well, so there was nothing to copy to my MP3 player. Was this the anticipated result? Perhaps I have the syntax wrong.

-LaRoche

[localhost:~/Desktop/scratch/Live At Legends] laroche% ls -l "Track 01.mp3"/rsrc
-rw-r--r-- 1 laroche staff 0 Feb 13 22:37 Track 01.mp3/rsrc
[localhost:~/Desktop/scratch/Live At Legends] laroche% ls -l "Track 01.mp3"
-rw-r--r-- 1 laroche staff 6364538 Feb 13 22:37 Track 01.mp3
[localhost:~/Desktop/scratch/Live At Legends] laroche% rm "track 01.mp3"/rsrc
[localhost:~/Desktop/scratch/Live At Legends] laroche% ls
Track 02.mp3 Track 04.mp3 Track 06.mp3 Track 08.mp3 Track 10.mp3
Track 03.mp3 Track 05.mp3 Track 07.mp3 Track 09.mp3 Track 11.mp3

LaRoche,

curious, rm'ing a file's /rsrc doesn't delete it, but it changes it's permissions to no acess for nobody! that makes it hard to see, but it's there.

% rm song.mp3/rsrc
% ll song.mp3/rsrc
ls: song.mp3/rsrc: No such file or directory

% ll song.mp3
---------- 1 merv wheel 4.5M Feb 16 21:48 song.mp3

good thing we tested and didn't unleash zapRez on your collection, eh?

so, it's prolly a bad thing to rm file/rsrc. we should take a different approach.

what we really want to do is empty the file's resource fork, make it zero length.

we saw that a mv filea to fileb on the same volume merely renamed its directory entry, but a mv to a different volume stripped the rez fork.

what would a cp to the same volume do?

% ll song.mp3
-rwxr--r-- 1 merv wheel 4.5M Feb 16 21:56 song.mp3*

% ll song.mp3/rsrc
-rwxr--r-- 1 merv wheel 41k Feb 16 21:56 song.mp3/rsrc*

% cp song.mp3 foo.mp3
% ll foo.mp3
-rwxr----- 1 merv wheel 4.5M Feb 16 21:57 foo.mp3*

% ll foo.mp3/rsrc
-rwxr----- 1 merv wheel 0 Feb 16 21:57 foo.mp3/rsrc*

> then mv back to original?

% mv foo.mp3 song.mp3
% ll song.mp3
-rwxr----- 1 merv wheel 4.5M Feb 16 21:57 song.mp3*

% ll song.mp3/rsrc
-rwxr----- 1 merv wheel 0 Feb 16 21:57 song.mp3/rsrc*

that's the _effect_ we're looking for. now to test that it's still a valid, playable mp3 file...

that makes scripting a little harder. now, if you had a nice (let's see, 3,000 songs, 3MB per song) 9GB target drive, it would be considerably easier. but let's assume you don't and we need to solve this in the available space.

% cp a b # shave off rez fork
% mv b a # rename file back to original

that's all for now. i hope to find a better solution.

MervTormel,

This last message has left me a little confused. I think you are still a few steps ahead of me as to what exactly is going on. I have a few questions that might help me out. When I do a ls -l file.mp3/rsrc and I get 0 bytes, is this a good thing or does it mean hidden files will be created when the file is moved to another volume? (BTW I have yet to find a file that said anything other than 0 bytes when this ls -l m/rsrc was launched on it.) Once the file has been moved to my MP3 player and hidden files are created, can these files be manipulated just as any other files? In other words, is there any way we can execute a rm command on the hidden files only, or will new forks just be created in their place? I also don't know what you mean when you say that (rm file.mp3/rsrc) doesn't delete the files, only changes their privledges. I haven't found any of the files I lost in this manner (in my scratch disk) using the "ll" command as you suggested (how is "ll" different from "ls" anyway). I am still under the belief that they were actually deleted. Also, if we do create some sort of script to fix the problem, will it be unleashed on the files in my MP3 player, or in my Itunes library?

Grateful for your patience, -LaRoche

LaRoche,

Forget all that stuff. I think we need to go subatomic. Let's try a test so that we can find an appropriate solution. remember, i don't have your mp3 player, so i rely on you for reporting accurate results.

Clear your mp3 player or whatever you have to do to see the results of two different kinds of copy.

Get two mp3 files to a scratch area, name them so that we know they are test files and we know they aren't on the player.

Do they have rez forks? What type of files are they? itunes or quicktime?

- Finder copy one to the player

- Shell copy the other to the player, like so

% cp /pathto/testfile.mp3 /pathto/mp3player/testfile.mp3

What are the results on the player?


we can't mess with zapRez script because of undesirable results.

re: ll, it's just an alias to an ls command with switches. It's just a shorthand expression, e.g.,...

% alias ll
ls -FlAh

If you've installed the Apple Developer Tools there is a neat little program called SplitForks that can be found here: /Developer/Tools/SplitForks. The program works as follows:

Welcome to Darwin!
Tue Feb 19 02:32:24 EST 2002
Chemistry professors never die, they just fail to react.
[localhost:~] hs2admin% cd desktop/TestFolder/
[localhost:~/desktop/TestFolder] hs2admin% ll
total 136
drwxr-xr-x 3 hs2admin staff 58 Feb 19 02:31 .
drwx------ 19 hs2admin staff 602 Feb 19 02:31 ..
-rw-r--r-- 1 hs2admin staff 8 Feb 19 02:31 TestFile
[localhost:~/desktop/TestFolder] hs2admin% ll TestFile/rsrc
-rw-r--r-- 1 hs2admin staff 332 Feb 19 02:31 TestFile/rsrc
[localhost:~/desktop/TestFolder] hs2admin% SplitForks TestFile
[localhost:~/desktop/TestFolder] hs2admin% ll
total 144
drwxr-xr-x 4 hs2admin staff 92 Feb 19 02:33 .
drwx------ 19 hs2admin staff 602 Feb 19 02:31 ..
-rw-r--r-- 1 hs2admin staff 414 Feb 19 02:33 ._TestFile
-rw-r--r-- 1 hs2admin staff 8 Feb 19 02:31 TestFile
[localhost:~/desktop/TestFolder] hs2admin%

This is the process that OSX uses to move files to and forth from non HFS systems. Note the second file with the "._" prefix. There is some documentation about this in the OSX System Overview (in the deveoper tools documenation and on Apple's website). I would assume there is another command to rejoin the 2 files but perhaps not as Apple seems to be phasing out resource forks. I'll leave it to Merv to come up with a good script to rm the offending files. Please post it for posterity.

Hugh

thanks, hugh, that may be useful. i poked around in /devo/tools but nothing leaped out at me for laroche's problem.

and that /devo/tools contains some deprecated products, i dint wanna fool with them as an ultimate solution.

i'm afraid we're lost regarding laroche's problem, and until we can see intimate details of the test, we don't know what's going on.

in particular, do his mp3's have rez forks on disk? are they of type QT or itunes? does the finder copy incarnate the rez forks? does the shell copy strip the rez forks?

we need meticulous test data from laroche, or someone with problems of the same ilk to corroborate, before we can crack this nut.

regards, -mt

Merv,

Finder - will copy resource forks if they are available but does not create them. Will re-attach a StripForks stipped fork if it finds one but only if the file is moved. In limited testing the file resplits when moved back to a location where it was split in two. I'm guessing this behavior is stored in the metadata of the resource fork but that requires more testing.
cp - strips resource forks into the ether
mv - strips resource forks into the ether when moving across file systems (will only change the pointer and keep the fork if you stay on the same system)
ditto - strips resource forks into the ether
ditto -rsrc - copies resource forks if they are available
zip - strips resource forks into the ether

This is all from Apple's docs and personal testing.

A quick thought would be StripForks, wildcard and rm ._, then transport the items to the mp3 player.

Please note - the mp3 player will need to be erased to get all of the ._ files off it as the finder will merge these with like named files when it finds them.

Hope this helps some more,
Hugh

ps - agree with you an apple's tools, there seem to be some issues. This one seems to work and is well utilized by apple for cross file system moves. There are no man pages however. Your guess is as good (probably better) as mine as to what the flags are.

Merv,

I've been trying to work out a shell script to solve this problem. I keep getting hung up with the syntax of rm. This is the script: #!/bin/sh
##
# NukeForks.sh
# SplitForks and Nuke the ResourcFork.sh
# Get rid of resource forks. This script requires
# the developer tools to be installed and in your path.
#

foreach file
SplitForks "${file}"
rm -f "${._file}"
end
I can't seem to get the syntax to rm only the ._ file. Do you have any thoughts on how to do this? I've played with it every way to Sunday and I currently have 4 big books on my desk. This is driving me nuts. :confused:

Thanks,
Hugh

hugh, thanks. hope you didn't pass a stone, but you might after this...

your example above with splitforks TestFile left out one important test. that is the test that the original file is barren of a rez fork. it isn't. or, mine aren't. splitforks makes jolly good ._files, but the "business" files still have their meaty bits.

second, your script's rm problem is that there is no variable name ._file

you need to reference something with the value of $file prepended with '._'

set fs="._$file"; echo $fs
._theFile

but you might as well write a script that says touch foo; rm foo because that's all nukefork.sh does.

except nukefork.sh does it's dirty business with extreme prejudice. i'd take that rm -f out and replace it with echo "here's the ${fs} i was gonna nuke" until you see good results.

mi dos centavos.

Ouch. I should have checked the finder. The files get larger after splitting the forks. And as you said the original resfork is still there. Way back to the drawing board.

Thanks,
Hugh

put yer etch-a-sketch away, hugh. i found one link to google: [ "mac os x" splitforks ] at arstechnica

http://www.arstechnica.com/reviews/01q2/macos-x-final/macos-x-15.html

and there i gleened a technique to zero the resource fork of a file. it works:

% ll */rsrc
-rw------- 1 merv wheel 286 Feb 19 15:05 b/rsrc
-rw------- 1 merv wheel 286 Feb 19 15:05 c/rsrc

% touch foo

% cp foo b/rsrc

% ll */rsrc
-rw------- 1 merv wheel 0 Feb 19 15:18 b/rsrc
-rw------- 1 merv wheel 286 Feb 19 15:05 c/rsrc
-rw-r----- 1 merv wheel 0 Feb 19 15:18 foo/rsrc

just copy a zero length file to the /rsrc of the target file.

caveat for LaRoche: even this may not be a cure for you. for we don't know if your rez forks will automagically reappear when they are finder copied to the player. still need to see the results of your testing.

Merv,
I think this is the test that you had in mind. My conclusion is that the hidden files are a clear by-product of the drag and drop process. Hope this helps you guys out. Thanks -LaRoche

Here is the test.
I have reformatted my MP3 player in OS9 as a DOS formatted disk. I imported 4 new files in Itunes in OSX. I have draged them to a scratch folder on my desktop. They have iTunes icons in the folder, and this is what they look like in the shell.

[localhost:~/Desktop/Scratch] laroche% ls -l test1.mp3/rsrc
-rw-r--r-- 1 laroche staff 0 Feb 19 21:23 test1.mp3/rsrc
[localhost:~/Desktop/Scratch] laroche% ls -l test2.mp3/rsrc
-rw-r--r-- 1 laroche staff 0 Feb 19 21:24 test2.mp3/rsrc
[localhost:~/Desktop/Scratch] laroche% ls -l *.mp3/rsrc
-rw-r--r-- 1 laroche staff 0 Feb 19 21:23 test1.mp3/rsrc
-rw-r--r-- 1 laroche staff 0 Feb 19 21:24 test2.mp3/rsrc
-rw-r--r-- 1 laroche staff 0 Feb 19 21:21 test3.mp3/rsrc
-rw-r--r-- 1 laroche staff 0 Feb 19 21:25 test4.mp3/rsrc

I am going to finder copy test1.mp3 : MV test2.mp3 : CP test3.mp3
I am also going to finder MOVE test4.mp3
I recieved an error with the mv command. The file made it to my MP3 but was not deleted from the scratch folder.

[localhost:~/Desktop/Scratch] laroche% mv test2.mp3 /Volumes/STUDIO20/test2.mp3
mv: /Volumes/STUDIO20/test2.mp3: set owner/group: Invalid argument

These are the results of the three files after they have been transferred to the MP3 player. Note: the file sizes of the .mp3 files remain identical. Snax shows only a hidden files being created on Test1.mp3 and test4.mp3, the finder transfers.

[localhost:/Volumes/Studio20] laroche% ls -l *.mp3
-rwxrwxrwx 1 laroche wheel 9247987 Feb 19 21:23 test1.mp3
-rwxrwxrwx 1 laroche wheel 7411585 Feb 19 21:37 test2.mp3
-rwxrwxrwx 1 laroche wheel 5970154 Feb 19 21:39 test3.mp3
-rwxrwxrwx 1 laroche wheel 8770991 Feb 19 21:25 test4.mp3
[localhost:/Volumes/Studio20] laroche% ls -l*.mp3/rsrc
ls: No match.
[localhost:/Volumes/Studio20] laroche% ll test1.mp3/rsrc
ls: test1.mp3/rsrc: Not a directory
[localhost:/Volumes/Studio20] laroche% ls -l test2.mp3/rsrc
ls: test2.mp3/rsrc: Not a directory
[localhost:/Volumes/Studio20] laroche% ls -l test3.mp3 /rsrc
ls: /rsrc: No such file or directory
-rwxrwxrwx 1 laroche wheel 5970154 Feb 19 21:39 test3.mp3
[localhost:/Volumes/Studio20] laroche% ls -l test3.mp3/rsrc
ls: test3.mp3/rsrc: Not a directory
[localhost:/Volumes/Studio20] laroche% ls -l test4.mp3/rsrc
ls: test4.mp3/rsrc: Not a directory

LaRoche, a most excellent and thorough test. i thank you. and i think you answered your questions.

finder copy to a DOS formatted disk too smart for it's own good; it bursts files to rez and data forks at the target. bad, in this case.

the last part of your test isn't valid. there would be no rez forks on the DOS drive. DOS filesystem thinks your trying to access a directory. but the rez forks are dutifully bursted into your unwanted files, right?

so, the solution is to script a nice shell copy to your player. this'll be pretty cool because perhaps you could wrap a nice cocoa applescript around it, and a daemon to automagically detect yer player, and present you with a standard playlist or allow you to customize it. nipples tingling. not really. but, you could roll yer own interface to update your player, after we get your problems solved, that is.

the next step is to see your source structure for your 3K mp3 files (just a sample, please) and your target structure.

what does the source look like, .e.g., genre/artist/album/song

% ll /Volumes/music/SurfRock/The\ Ventures/Wild\ Again\!/
total 9.7M
-rwxr--r-- 1 merv wheel 3.9M Dec 29 18:07 James Bond Theme*
-rwxr--r-- 1 merv wheel 2.5M Dec 29 18:07 Midnight In Moscow*
-rwxr--r-- 1 merv wheel 3.4M Dec 23 22:48 The Lonely Surfer*


and what is the source required to look like?

then, we'll have a go-go at a prototype script.

note that the mv command has special implementation for moving files from one volume to another. in that case it acts as a copy src targ; rm src

odd that mv error. i guess it was trying to chown the target file, but it's on a foreign filesystem, so it pooped its pants. it deliberately leaves the source intact in this case.

so, is mv not smart enough to consider it's target?

Merv,

I just got a book on Unix to aid me with the scripting. I'll try to keep up. My MP3s are categorized on my local harddrive at this

/Volumes/laroche/music/wuxia/artist/album/files.mp3s

On the root level of my MP3 player they look like this..

/artist/album/files.mp3

A quick question about the cp command. I know that cp -r does a recursive copy from a directory, but what if I only want the subdirectories. For instance a cp -r /wuxia/ /Volumes/archos20/ will copy all of my files, but with the Wuxia directory, which I don't want on my mp3 player.

-LaRoche

Merv,

I think I have found a solution that is much simpler than writing a shell script. There is a freeware java-based file manager called "muCommander" that easily copies files to my MP3 player without rez forks. Thanks for all of your help, I learned a lot about OS-X with your assistance.


-LaRoche

missed your post about the cp -r dilemma (there's always a way). hope your utility works well. if you want to come back and hack away at the script, jest hollar.

regards, -mt