| Author |
Message |
| < Public Announcements ~ Emergency Upgrade: server4: 30 Minutes |
|
 Posted:
Thu Jan 13, 2005 3:48 pm
|
|
|
|
|
We will performing emergency maintanence on server4 in approximately 30 minutes: Thursday, January 13, 2005 at 12:15 AM.
This upgrade is for a critcal kernel upgrade, outage should last no longer than 30 minutes.
If you have any questions please submit a support ticket via our help desk: http://www.crucialparadigm.com/support/
Regards,
Aaron |
Last edited by Aaron on Sun Jan 23, 2005 3:43 am; edited 1 time in total |
|
|
|
|
 |
|
| Posted:
Thu Jan 13, 2005 5:00 pm
|
|
|
|
|
| First attempt was unsuccessfull a total downtime of 2 minutes was recorded. We are going for another reboot within the next 10 or 20 minutes. |
|
|
|
|
|
|
|
| Posted:
Thu Jan 13, 2005 6:19 pm
|
|
|
|
|
It appears after out latest attempt the grub boot loader is not coming up. We are in the process of using rescue CDs to bring up the server as soon as possible.
We apologise for the extended downtime.
Regards,
Aaron |
|
|
|
|
|
|
|
| Posted:
Thu Jan 13, 2005 8:22 pm
|
|
|
|
|
We have received unfortunate news from the datacenter, it appears that the partition table on the primary drive has been corrupted.
We are having a data corruption specialist analyse as the drive as we speak, if they find the drive is recoverable we will attempt to get it recovered, if not, then we wil restore from backups.
Downtime for this will range from 6-24 hours.
Please accept our sincere apologies for the downtime, we are working around the clock and have several staff members currently working on this.
Regards,
Aaron |
|
|
|
|
|
|
|
| Posted:
Thu Jan 13, 2005 9:23 pm
|
|
|
|
|
Update:
The primary drive has failed and is unrecoverable, we are replacing the drive with a new 200GB SATA drive, and the server should be up within 1 hour.
After the server comes backup online accounts will start being recovered alphabetically. Estimated time to recover all accounts is 24 hours.
If you have any questions please submit a support ticket.
We do appologise for the downtime and we will look into compensation once the accounts have been restored.
Regards,
Aaron |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 1:05 am
|
|
|
|
|
Update:
The server is now back online. We are in the process of configuring the server now and will start restoring accounts shortly. |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 3:20 am
|
|
|
|
|
Update:
Account restoration has started. |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 6:52 am
|
|
|
|
|
Update:
All a* accounts have been restored. |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 6:57 am
|
|
|
|
|
Update:
All accounts starting with b have been restored. |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 7:55 am
|
|
|
|
|
Update:
All accounts starting with c have been restored.
Private Name Servers
Please note that all users with private name servers on server4 should submit a support ticket with their name servers and IP addresses, this will speed up the restore process. |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 2:15 pm
|
|
|
|
|
We have good news, we do have up to date backups of all home directories as of the entire mysql as of the time the service crashed. We will first restore older backups (from the 9th), and update those with the more recent backups once all accounts have been restored.
We are now up to the letter m on our backups, about half way through the first set of backups. |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 3:45 pm
|
|
|
|
|
| We are currently working on restoring backups starting with p. |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 6:41 pm
|
|
|
|
|
| Restoration of r has been completed, and s will be starting shortly. |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 7:42 pm
|
|
|
|
|
| S has been completed we are now starting on t. |
|
|
|
|
|
|
|
| Posted:
Fri Jan 14, 2005 8:01 pm
|
|
|
|
|
| Restoration of t complete, working on x now. |
|
|
|
|
|
|
|
