Check web page download times - use curl?

curl will write the download time (to the nearest microsecond) to stdout. You would have to redirect the number somewhere, like append it to a file. Adding up a list of numbers should be an easy array/list operation in almost any scripting language.
If this is the same page, getting the file list should be very easy. If you are testing a multitude of pages it will be quite difficult.

However, tell us why you are doing this and maybe a better solution will appear. Download times are very ethereal and depend on a huge number of factors. Images recycled between pages are typically downloaded once and cached, so the reload time is faster. Same for style sheets. A lot of hosts charge for bandwidth, so it doesn't make much sense using up your bandwidth allocation testing it. If you are on a shared host the available speed will vary greatly depending on the other sites usage.

Another thing that curl will not address is rendering time/experience -- if you test in real browsers you can see things like jumpy rendering because you forgot to specify image sizes, or things that don't happen until javascript's onLoad event happens, etc.

You can also just precede the the curl with the "time" command:
pink% /usr/bin/time curl -O "http://heanet.dl.sourceforge.net/sourceforge/gimp-app/Gimp-2.2.6.dmg"
% Total % Received % Xferd Average Speed Time Curr.
Dload Upload Total Current Left Speed
100 42.0M 100 42.0M 0 0 1623k 0 0:00:26 0:00:26 0:00:00 1673k
26.57 real 0.16 user 1.48 sys


The first line is the command, lines 2 and 3 are the output from curl, and the last line is the output from the time command.

cheers, pink

Thanks for the replies. I need to test only one page, downloading all the elements (images, css etc.); bandwidth is not a problem. It would be nice, as somebody said, to be able to use a browser to include rendering time (but I don't want to use the browser cache as I want to simulate a fresh user) in the total download times but I have absolutely no idea how to do that (Applescript maybe?).
You say that curl write the download times to stdout, as my knowledge of this stuff is nearly zero, could you please show us an example of a script that downloads a page (or image or whatever) and writes the download time to a file? I may then figure it out how to apply that to the rest of the elements.

Many thanks

Mike

What you need to do is direct the main output (the downloaded file) to one place, the program's output (statistics) to another and the special output (the time) to a third. We do this with a combination of program and shell redirection.

try this:

curl -s -w "%{time_total}\n" -o /dev/null http://www.macosxhints.com/index.php >> count-this

Dissection:

curl : invoke the main command
-s : Silent, supress the program's normal output (the human-readable statistics)
-w "%{time_total}\n" : Write a string, consisting of the variable containing the total operation time followed by a newline character, to stdout.
-o /dev/null : dump the main output to a file, in this case the unix trash as we don't care about the content.
http://....... : the fully qualified url we are interested in.
>> count-this : take stdout (from -w, above) and append it to the file "count-this" for later computation.

Nearly there! Thanks acme.mail.order the code you suggested worked fine so I tried to use a configuration file to get the different page elements writing this:

curl -s -w "%{time_total}\n" -o /dev/null -K "conf" >> count-this

where conf is the file that contains the urls to get in the format:

url = "http://www.webaddress.com"
url = "http://www.webaddress.com/stylesheet.css"
.....

curl seems to read the conf file ok, but the only problem is that the code is dumping the content of the second file (in this case stylesheet.css) in the count-this file. What am I doing wrong?
And one final question: how do I add up the different times that will appear in count-this? I was thinking of using something like head or grep - any tips?

Thanks

Mike

try putting ALL the options in the conf file, reducing your command to "curl -K "conf" >> count-this
The second time through is the second line of the conf file, which simply says get this url and dump it to stdout.

head and grep don't do math. You will need to either:
a) import the file into a spreadsheet and add up the column, or
b) write a program to read the file, loop through the contents and add up a variable. I can't think of any scripting language that won't do this in about 10 lines or less. You have bash, tcsh, php, perl, python and AppleScript preinstalled.

Works perfectly, thanks! Re the calculations I'll use a speadsheet.

Cheers

Mike