PHP question

[lovingboth]

I want one of my websites to get data from another website. Every 12 hours, cron will run a PHP script to snaffle it and stick it in a database for later use.

Sometimes, for whatever reason, the snaffle may not work. (The other site may be down, or busy or there may be routing problems or whatever.)

The obvious thing to do is say 'try snaffling until success', but I'm reluctant to alter the php.ini setting that says scripts run for a max of 30 seconds, so...

... is there a good alternative to having a PHP script that says in effect

Is the data already snaffled?
If not, get it.

and getting cron to run this at, say, 00:00, 00:15, 00:30, 00:45 to give four chances of a successful snaffle? The check would be a really quick SQL query, so if it succeeded the first time, the subsequent runs would be very short.

Comments

[fruit-boy.livejournal.com]

Go snaffle data
Has it worked???
- yes......exit(0)
- no.......write entry into crontab to try again in 2(?) mins then exit(0)

[mattp.livejournal.com]

As a failure would likely be temporary, and to save the need to tidy up the cronjob upon success, adding an entry as an atjob would be better.

Seems fine in theory, though you might need to have some form of concurrency detection in case one job is "set back" so many times it runs over into the next cron execution.

A way around this might be not to enter the job in atd if it's closer than $threshold to the next scheduled cron execution. (And perhaps to send off a warning email if necessary)

[fruit-boy.livejournal.com]

A completely different solution would be to have the script run every 5 minutes and to first check a control file for the last time data was sucessfully snaffled. If this is less than 24 hours ago then simply exit. If not, attempt to snaffle data and if sucessful write the current time back to the control file.

[pavlos.livejournal.com]

I would suggest keeping the limit on PHP run time, but having some other agent, e.g. cron->wget call your scrip repeatedly during the day. If the data is already fetched, the PHP script should do nothing.