I'm writing a script to consume an XML feed provided by a third party. I'm having problems downloading the files listed inside it but the provider swears there's no problem.
The files are usually around 700kb, but between 20% and 80% of the downloads just stall mid-way, and then after 10 minutes of no packets being received, I receive a RESET packet (according to wireshark).
In order to test this I have a very simple shell script:
FILE="http://www.domain.com/uploads/g4nsloshv0ppubwxxxulr4bx/an-example-file.pdf"
OUTPUT="output.pdf"
curl $FILE > $OUTPUT
I've run this many times on two machines in my office, as well as from an Amazon EC2 instance, and in all cases I see a mixture of very quick successes and also progress just halting at some mid-way point (different each time) and then never completing.
The success rate varies hugely - I might get 10 successes and then 10 stalls in a row.
I'm hoping someone can tell me if this is a common symptom for an overloaded .NET server (IIS I presume?) or whether you think there's anything I can do to prove the problem more strongly. Or indeed if there's something different I can do at my end.
-
Have you tried different using a different user agent?
Jamie : I haven't tried changing the user agent in cURL, but I see the same problems when downloading manually with Firefox so I'm pretty sure the user agent isn't affecting it.From gaz -
Watch it with Fiddler. You should see whats going on at the HTTP level, alongside wireshark which is showing you the traffic.
You may find an HTTP request/response that is causing the behavior.
Look at the headers, check content-length, etc -- looking for anything that could cause the behavior
See if the response looks right, etc
That should help you narrow down and at least understand the full scope of the issue. Then I'd bet you'll find something on your end or have the evidence to prove its them
From Taylor
0 comments:
Post a Comment