High Loads - DOS Attack (Resolved)
  • Priority - Critical
  • Affecting Server - cpanel42.onlinemountain.com
  • This server is under high loads and we are working to resolve.

  • Date - 07/05/2017 16:15 - 07/06/2017 10:03
  • Last Updated - 07/05/2017 16:16
High Loads - DOS Attack (Resolved)
  • Priority - Critical
  • Affecting Server - ED-SH-CP7
  • This server is under high loads and we are working to resolve.

  • Date - 07/05/2017 16:15 - 07/06/2017 10:03
  • Last Updated - 07/05/2017 16:15
Server Restart in progress (Resolved)
  • Priority - High
  • Affecting Server - cpanel41.onlinemountain.com
  • This server requires a reboot - now in progress....

  • Date - 07/01/2017 10:23 - 07/01/2017 10:36
  • Last Updated - 07/01/2017 10:24
MySQL Server down (Resolved)
  • Priority - High
  • Affecting Server - cpanel41.onlinemountain.com
  • MySQL server cannot strat due to the mysql data drive mounted as read only.  We are investigating the cause and looking for a resolution.

    We were able to get the drive back online and MySQL is now operating normally with no errors.
    --

    Reopeing this issue as some users are experiencing authentication errors.
    We are working with cpanel to resolve.

  • Date - 06/24/2017 15:40 - 06/26/2017 11:35
  • Last Updated - 06/25/2017 21:22
Apache build failed (Resolved)
  • Priority - High
  • Affecting Server - cpanel42.onlinemountain.com
  • Rebuild of apache has failed and we have escalated the issue to cpanel support

  • Date - 06/12/2017 21:11 - 06/12/2017 22:09
  • Last Updated - 06/12/2017 21:12
High Loads (Resolved)
  • Priority - High
  • Affecting Server - cpanel41.onlinemountain.com
  • We are seeing extremely high loads on this server and are investigating the cause.

  • Date - 05/17/2017 10:30 - 06/12/2017 21:12
  • Last Updated - 05/17/2017 10:31
High Loads (Resolved)
  • Priority - High
  • Affecting Server - cpanel42.onlinemountain.com
  • This server is experiencing high loads and we are currently investigating possible causes.

    UPDATE
    -we have found some IP addresses that are attacking some wordpress sites on the server and are mitigating the effects.

    UPDATE
    -this server is once again under high loads.

    UPDATE
    -we have found the issue - a huge ip.pag file from the DDOS attack previously reported was causing high disk IO in apache which was causing a cascading effect and failure.
    -should be ok now.

  • Date - 05/15/2017 14:50
  • Last Updated - 05/16/2017 17:22
root compromised - backup/restore in progress (Resolved)
  • Priority - High
  • Our security scans have revealed that this server is root compromised.  As such, we have started provisioning a new server (with a newer OS) and will:
    1. Backup your cpanel account
    2. Take the current suitland.webnethost.net server offline.
    --during this time, your site and email will be offline.
    3. Once the new server is online, restore your cpanel account to the new server.  Once your account is restored, your website and email will be online once again.
    4. There will be no IP change to your domain.

    This process may take 1-3 hours with intermittent connectivity to your site and email.  This is an urgent matter and must be addressed immediately.

    Please check back for updates.

    ---

    Apr 24 - 17:36 EST (GMT -5:00)

    The new server is set up and the backups are complete. I will be taking the compromised server down shortly and putting the new server in it's place. I will then run scripts on the new server that will install your services. After the post-install scripts are finished I will begin restoring your accounts to their proper IP address.

    I will update you on the progress as best I can. Please be patient, however, as this will take some time.

    ---

    Apr 25 - 10:00 EST (GMT -5:00)

    Account restores are complete - cpanel account passwords have all been reset.  We will advise your new password shortly.  Some MIVA Merchant sites on this server are not functional - we are working with MIVA support to get the miva components re-installed.

    ---

    Apr 25 - 13:05 EST (GMT -5:00)

    Please note that all cpanel passwords were reset to protect the integrity of your account.  Please login to your client area at https://www.entirelydigital.com/manage and then click on your services icon, then drill in to the account that is on this server.  From there you will be able to reset your cpanel password from the left menu.

  • Date - 04/24/2017 15:54 - 05/15/2017 14:50
  • Last Updated - 04/25/2017 13:06
Auto SSL Failure (Resolved)
  • Priority - High
  • The AutoSSL feature of this server has failed and certificates which expired today have not been renewed.  We are investigating the cause and hope to have a resolution shortly.

    This issue has been resolved and the SSL certificates are queued at Comodo for installation.

  • Date - 02/07/2017 16:24 - 04/24/2017 15:55
  • Last Updated - 02/07/2017 18:25
Drive Maintenance (Resolved)
  • Priority - High
  • potomac.webnetservers.com is displaying some issues with I/O. We require a maintenance window with the server offline for 1-4 hours to clone the primary drive.

  • Date - 01/16/2017 22:00 - 01/17/2017 04:50
  • Last Updated - 01/16/2017 20:48
Server unresponsive (Resolved)
  • Priority - Critical
  • This server has become unresponsive and the issue is being actively investigated.

    --

    This isssue is now resolved.

  • Date - 12/29/2016 13:20 - 12/29/2016 13:43
  • Last Updated - 12/29/2016 13:43
SMTP Slowdown (Resolved)
  • Priority - Critical
  • Affecting Server - cpanel43.onlinemountain.com
  • Because of a spamming account, this server's SMTP queue is being cleared of spam messages
    There are several hundred thousand messages to clear.
    Once clear we will divert outbound mail via an alternate IP.

  • Date - 12/12/2016 18:22 - 12/13/2016 11:03
  • Last Updated - 12/12/2016 18:22
SMTP Slowdown (Resolved)
  • Priority - Critical
  • Affecting Server - cpanel42.onlinemountain.com
  • Because of a spamming account, this server's SMTP queue is being cleared of spam messages
    There are several hundred thousand messages to clear.
    Once clear we will divert outbound mail via an alternate IP.

  • Date - 12/12/2016 18:21 - 12/13/2016 11:03
  • Last Updated - 12/12/2016 18:22
Hardware Maintenance Window (Resolved)
  • Priority - High
  • A hardware maintenance window is required for this server in the times shown
    All services on this server will be unavailable during the maintenance window as the server will be powered off
    Actual time will be approximately 1 hour.

  • Date - 12/06/2016 21:00 - 12/07/2016 14:13
  • Last Updated - 12/06/2016 16:02
Hardware Maintenance Window (Resolved)
  • Priority - Medium
  • A hardware maintenance window is required for this server in the times shown
    Dec 1, 01:00-05:00 EST (GMT -5:00)
    All services on this server will be unavailable during the maintenance window as the server will be powered off
    Actual time will be approximately 1 hour.

  • Date - 12/01/2016 01:00
  • Last Updated - 12/06/2016 16:00
IIS6 WH-01 Being Moved to New HOST (Resolved)
  • Priority - Medium
  • Affecting Server - ED-DNP01
  • Our legacy IIS 6 server is being moved to a new host for improved stability.  During this move, websites on this server will be inaccessible.
    Move time is estimated at 15minutes.

    Migration complete

  • Date - 11/19/2016 17:10 - 11/19/2016 17:39
  • Last Updated - 11/19/2016 17:35
Server Loads Extremely High (Resolved)
  • Priority - High
  • We are investigating extremely high loads on this server.

  • Date - 11/16/2016 16:28 - 11/16/2016 17:56
  • Last Updated - 11/16/2016 16:28
Kernel update and reboot (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel41.onlinemountain.com
  • System kernel update in progress + reboot of server

    --

    FSCK in progress forced - startup delay.

    ---

    Reboot complete.

  • Date - 09/03/2016 11:16
  • Last Updated - 09/03/2016 12:11
Server Load High (Resolved)
  • Priority - High
  • Affecting Server - cpanel43.onlinemountain.com
  • The high load on this server is being investigated.  While our investigation proceeds, port 80 has been blocked to allow our technicians the ability to work in a timely manner.  All other ports remain open (POP, IMAP, FTP, SMTP, Cpanel)

    ---

    Apache / PHP rebuild in progress.

    ---

    Build complete - we are now doing a kernel update.

    ---

    System being rebooted now - new kernel installed and updates complete.

    ---

    We've found the root cause of this issue and have resolved it.  Loads are now normal and the server is responsive.  We are monitoring it continuously.


  • Date - 09/02/2016 13:07 - 09/03/2016 11:13
  • Last Updated - 09/02/2016 17:36
Incoming email problem (Resolved)
  • Priority - Critical
  • Affecting Server - ED-SH-CP7
  • Currently, all email is queued and not being delivered to individual user mailboxes.  This is an issue with a failed cpanel update.
    We are investigating and hoping to have a resolution shortly.

    ALL DELIVERY ISSUES HAVE NOW BEEN RESOLVED.

  • Date - 08/12/2016 16:28 - 08/12/2016 20:11
  • Last Updated - 08/12/2016 20:11
Hosted Exchange Update (Resolved)
  • Priority - Medium
  • Affecting System - Hosted Microsoft Exchange
  • We have scheduled some mailbox server updates - Microsoft Hosted Exchange 2010
    These are general service and stability updates for Exchange and Windows Server as well as funcitionality updates for OWA (Outlook Web Access)

    Updates started

    1st reboot in progress....

    Reboot complete.
    Starting second set of updates

    2nd reboot in progress....

    Exchange updates in progres - your mailbox will be unavailalbe until completion...

    Mailboxes now available

    Further updates may make your mailbox unavailable for a few minutes....

    Last set of updates require a reboot to complete.  Reboot in progress....

    UPDATES COMPLETE

  • Date - 08/06/2016 08:00
  • Last Updated - 08/06/2016 12:04
cp.neiko.net - login unavailable (Resolved)
  • Priority - Medium
  • Affecting Server - VIPER-HELM-SH
  • We are currently experiencing login issues with http://cp.neiko.net - this is a licensing problem with the control panel server which we are working to resolve.

  • Date - 07/21/2016 11:39
  • Last Updated - 07/29/2016 14:10
cPanel servers (Resolved)
  • Priority - Critical
  • Affecting System - cPanel Security Team - CVE-2016-3714 ImageMagick
  • cPanel Security Team - CVE-2016-3714 ImageMagick

    Background Information

    On Tuesday, May 3 2016, ImageMagick announced a vulnerability in all versions of the ImageMagick software. ImageMagick is a software
    package commonly used by web services to process images.

    Impact

    One of the reported vulnerabilities can potentially be exploited for remote code execution (RCE).

    Releases

    ImageMagick has not released a fix, but plans to publish a new version of ImageMagic with the fixes soon. cPanel normally releases all builds at once in order to limit the ability to reverse engineer fixes. However, this vulnerability is already wildly known and we have seen reports of its use. In this instance, we plan to release builds as soon as they become available.

    At this time the following builds are available:
    11.56    11.56.0.13
    EDGE 11.55.9999.193
    CURRENT  11.56.0.13
    RELEASE  11.56.0.13

    We are currently updating all our cpanel servers to this build.  If you cannot access cpanel or WHM then a cpanel update is in progress.  Please be patient as the update completes.  

  • Date - 05/04/2016 12:18 - 05/04/2016 15:14
  • Last Updated - 05/04/2016 12:20
Update in progress (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel43.onlinemountain.com
  • cpanel43.onlinemountain.com is doing a cpanel udpate to 11.56 - as such cpanel and WHM are currently unavailable.

    Normal operations should resume shortly.

  • Date - 04/30/2016 11:43 - 04/30/2016 12:15
  • Last Updated - 04/30/2016 11:44
Server Inaccessible - Unscheduled Maintenance (Resolved)
  • Priority - Critical
  • Affecting Server - cpanel43.onlinemountain.com
  • We are investigating an issue with this server - updates to follow in this Network Issue notification

    --UPDATE--

    We see an issue with the installed kernel - we are reverting to the previous kernel and rebooting
    Investigation will continue.

    --UPDATE--

    Server is operational - we will be rebooting the server to a newer kernel later tonight.

    --UPDATE--

    Kernel has been updated correctly and verified - we are rebooting the server so that dependencies with CloudLinux function correctly.

    --REBOOT COMPLETE

    --UPDATE--

    We are once again noticing high loads on this server - invesigtating the issue - we will cycle through services and block specific ports to track down the culprit.

    --UPDATE--

    We are now running further updates to kernel and other drivers/cloudlinux in an attempt to narrow down the possible issue.

    --UPDATE--

    In order to expedite this process, we've had to block port 80 on this server as the update process was taking too long.  We have restarted the update process with port 80 blocked (low load) and hope to reboot the server shortly.

    --UPDATE--

    we have resolved the issue and loads are now normal.

  • Date - 03/06/2016 18:01 - 03/07/2016 16:16
  • Last Updated - 03/07/2016 16:16
Server Reboot (Resolved)
  • Priority - High
  • Affecting Server - cpanel41.onlinemountain.com
  • A kernel dependancy requires that we reboot the server for maximum stability.
    Reboot has begun and server should be back online in 5-10 min.

    --REBOOT COMPLETE

  • Date - 03/06/2016 20:15
  • Last Updated - 03/06/2016 20:29
SHA1 Certs - onlinemountain.com (Resolved)
  • Priority - Critical
  • Affecting System - SSL Certificates - SHA1 replacement
  • We are currently replacing all onlinemountain.com COMODO SSL certificates with SHA2 encrypted certificates.

    Prior to replacement you will get a warning in Google Chrome.

    Thank you for your patience

  • Date - 12/19/2015 12:04 - 03/06/2016 18:02
  • Last Updated - 12/19/2015 12:05
Core Infrastructure Upgrades (Resolved)
  • Priority - Medium
  • Affecting System - Core Network
  • We will be updating some core switches.  During the upgrades there may be some downtime you experience - Approximately 5 minutes while switchports come online.
    -This maintanance windows has been postponed.  A new item will be opened once the maintenance window has been determined.

  • Date - 12/06/2015 18:30 - 12/06/2015 20:59
  • Last Updated - 12/06/2015 17:31
Core Infrastructure Upgrades (Resolved)
  • Priority - Medium
  • Affecting System - Toronto DC
  • We will be updating some core switches.  During the upgrades there may be some downtime you experience - Approximately 5 minutes while switchports come online.
    -This maintanance windows has been postponed.  A new item will be opened once the maintenance window has been determined.

  • Date - 11/22/2015 21:00 - 11/22/2015 22:00
  • Last Updated - 11/22/2015 20:16
Apache 500 Errors - Cloudlinux (Resolved)
  • Priority - Critical
  • Affecting Server - cpanel42.onlinemountain.com
  • Most websites throwing 500 errors due to a Cloudlinux CageFS issue.
    We have escalted this to our Level3 support and hope to have a resolution shortly.

  • Date - 11/01/2015 14:38 - 11/01/2015 18:05
  • Last Updated - 11/01/2015 17:39
All 208.92.222.x IP network is down. (Resolved)
  • Priority - High
  • Affecting System - All 208.92.222.x IP network
  • Due to a DDOS in progress all 208.92.222.x IP network is down.
    We are working with our downstream provider to resolve the issue.

  • Date - 09/15/2015 19:56 - 09/15/2015 20:13
  • Last Updated - 09/29/2015 09:13
Reboot in progress (Resolved)
  • Priority - High
  • Affecting Server - cpanel41.onlinemountain.com
  • This server was under very heavy load - a full restart is in progress.
    - file system quotas check in progress (fsck) - this must complete before the server comes back online.

  • Date - 05/29/2015 16:02 - 05/29/2015 17:15
  • Last Updated - 05/29/2015 16:28
IMAP/POP logins failing (Resolved)
  • Priority - Critical
  • Affecting Server - cpanel41.onlinemountain.com
  • We are aware of the issue regarding this server.
    POP3 and IMAP connections may fail with an authenticator error.
    We are actively investigating this issue and should have a resolution shortly.

  • Date - 03/25/2015 16:18 - 03/25/2015 18:17
  • Last Updated - 03/25/2015 16:19
Server down (Resolved)
  • Priority - Critical
  • Affecting Server - cpanel42.onlinemountain.com
  • We are awere of the issue with this server and are actively investigating.

    --issue is now resolved.

  • Date - 03/20/2015 15:11
  • Last Updated - 03/20/2015 15:23
MAINTENANCE - HARDWARE Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel41.onlinemountain.com
  • We are currently doing a hardware upgrade on this server. During the upgrade no services are available.

  • Date - 02/25/2015 21:21 - 02/25/2015 21:44
  • Last Updated - 02/25/2015 21:22
PHP update in progress (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel41.onlinemountain.com
  • During updates websites will be unavailable.

  • Date - 02/23/2015 20:02 - 02/25/2015 21:42
  • Last Updated - 02/23/2015 20:02
Webmail/IMAP/POP Access down (Resolved)
  • Priority - High
  • Affecting System - Mail server - WEBM1 - TOR DC 1
  • We are seeing intermittent webmail issues with this server (https://webm1.entirelydigital.com) and are working to resolve.
    You may see continuing intermittent outages while we restart services and update core files.

  • Date - 02/06/2015 14:20 - 02/10/2015 10:23
  • Last Updated - 02/06/2015 14:21
POP/IMAP Webmail Authentication Failure (Resolved)
  • Priority - High
  • Affecting Server - cpanel41.onlinemountain.com
  • This server is refusing to authenticate POP, IMAP and WEBMAIL connections (via IMAP) once more.  We've escalated this to cPanel.net for immediate investigation and permanent resolution.

    ---

    This is now resolved but we are investigating the cause.

  • Date - 02/03/2015 08:06 - 02/03/2015 10:49
  • Last Updated - 02/03/2015 08:43
Apache/PHP recompile (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel41.onlinemountain.com
  • We are updating apache on this server - websites will be down for approximately 10-15 min.

    ---

    ETA changed - additional updates required for security reasons.

    ---

    Now complete - websites available.

  • Date - 02/01/2015 17:08
  • Last Updated - 02/02/2015 10:48
IMAP/POP/SMTP Authentication Failure (Resolved)
  • Priority - High
  • Affecting Server - cpanel41.onlinemountain.com
  • This server is experiencing IMAP/POP/SMTP Authentication Failure for most accounts.
    We are actively investigating this issue and hope to have a resolution shortly.

  • Date - 01/28/2015 10:31 - 01/29/2015 12:01
  • Last Updated - 01/28/2015 10:32
Server maintenance (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel42.onlinemountain.com
  • Server is currently undergoing maintenance - some hosting services will be affected during our upgrades

  • Date - 01/18/2015 07:12 - 01/18/2015 10:34
  • Last Updated - 01/18/2015 09:14
Server Maintenance (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel43.onlinemountain.com
  • Server is currently undergoing maintenance - some hosting services will be affected during our upgrades

  • Date - 01/18/2015 07:13 - 01/18/2015 09:34
  • Last Updated - 01/18/2015 09:14
Server under high load (Resolved)
  • Priority - High
  • Affecting Server - cpanel41.onlinemountain.com
  • This server experienced a system failure during updates and we had to gracefully restart it.  Unfortunately a forced filesystem check is in progress and must complete.
    Hopefully won't be much longer.

    --

    We are trying to deterimine what processes are causing high load - during our troubleshooting process, different services will become unavailable.

    --

    Currently httpd is turned off while we update cloudlinux cageFS - email and other services are up - this is required to ensure that cloudlinux LVE is working properly before we start httpd

    --

    Kernel updates are complete as well as updates to CloudLinux LVE - we're rebooting the server to load the new kernel
    We'll be watching the server carefully over the next hours to ensure the loads stay in check.

    --

    Server being monitored and processes are being kept in check

    --

    In cooperation with cpanel.net support we are close to a resolution regarding the performance of this server.  We appreciate your patience.

    --

    We've identified this is a DDOS attack and are taking steps to attempt to mitigate the attack

    --

    The DDOS seems to be targeted to specific customers - we are working to mitigate the attack however DDOS can be difficult to stop.

    --

  • Date - 01/13/2015 09:40 - 01/14/2015 09:18
  • Last Updated - 01/13/2015 19:50
Cloud Backup Server Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - OBS
  • Our cloud backup service is being updated to the latest build - during the upgrade offsite backup and online portal access is unavailable

  • Date - 01/09/2015 13:45 - 01/09/2015 14:00
  • Last Updated - 01/09/2015 13:46
Apache/PHP recompile (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel43.onlinemountain.com
  • PHP is being rebuilt (native 5.4) to include MBSTRING as it was not included in the original build.

  • Date - 12/08/2014 15:47 - 12/19/2014 12:31
  • Last Updated - 12/08/2014 15:48
Hosted MS Exchange Certificate Warning (Resolved)
  • Priority - Critical
  • Affecting System - Hosted MS Exchange
  • Clients are receiving invalid SSL certificate warnings in webmail and outlook - we are working with Godaddy SSL to re-issue the SSL certificate in question.

    UPDATE - Certificate has been rekeyed and installed - there are no furhter issues with webmail

  • Date - 12/08/2014 10:44
  • Last Updated - 12/08/2014 11:32
PHP update in progress (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel42.onlinemountain.com
  • PHP and Apache being updated to the latest build

  • Date - 12/06/2014 10:45 - 12/06/2014 11:39
  • Last Updated - 12/06/2014 10:45
PHP update in progress (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel41.onlinemountain.com
  • PHP and Apache being updated to the latest build

  • Date - 12/06/2014 10:45 - 12/06/2014 12:42
  • Last Updated - 12/06/2014 10:45
PHP update in progress (Resolved)
  • Priority - Medium
  • Affecting Server - cpanel43.onlinemountain.com
  • PHP and Apache being updated to the latest build

  • Date - 12/06/2014 10:44 - 12/06/2014 11:39
  • Last Updated - 12/06/2014 10:45
Cloud Backup Server Upgrade (Resolved)
  • Priority - Medium
  • Affecting Server - OBS
  • Update of Cloud Backup Server in progress.
    Disk Check in progress

    ---

    Disk check is still in progress but almost complete - should be operational shortly.

  • Date - 09/08/2014 16:08 - 09/09/2014 22:00
  • Last Updated - 09/08/2014 20:17
PHP update in progress (Resolved)
  • Priority - High
  • Affecting Server - cpanel41.onlinemountain.com
  • To address possible security vulnerabilities in php - we are updating PHP to 5.4.31 outside the normal maintenance window
    This is only a MID-LEVEL security issue.

  • Date - 08/19/2014 10:16 - 08/20/2014 12:32
  • Last Updated - 08/19/2014 10:17
Failed cpanel update being re-run (Resolved)
  • Priority - High
  • This server had a failed cpanel update causing some issues - we are re-running it now to stabilize the server.

  • Date - 08/18/2014 10:22 - 08/19/2014 10:16
  • Last Updated - 08/18/2014 10:23
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

  • Date - 07/30/2014 18:25 - 07/31/2014 10:22
  • Last Updated - 07/30/2014 18:25
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • Affecting Server - ED-SH-CP6
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

    -- update --
    We are now upgrading MySQL to 5.5 for compatiblity and upgrade paths to WHM 11.44

    -- update --
    We are now upgrading WHM to 11.44

  • Date - 07/26/2014 10:56 - 07/28/2014 15:29
  • Last Updated - 07/26/2014 12:58
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

  • Date - 07/23/2014 17:01 - 07/26/2014 10:56
  • Last Updated - 07/23/2014 17:15
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

  • Date - 07/23/2014 11:15 - 07/23/2014 12:21
  • Last Updated - 07/23/2014 11:14
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

  • Date - 07/16/2014 22:02 - 09/08/2014 16:08
  • Last Updated - 07/16/2014 19:02
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

  • Date - 07/16/2014 22:02 - 07/16/2014 19:02
  • Last Updated - 07/16/2014 19:02
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

  • Date - 07/16/2014 19:01 - 09/08/2014 16:08
  • Last Updated - 07/16/2014 19:01
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

  • Date - 07/16/2014 19:01 - 09/08/2014 16:08
  • Last Updated - 07/16/2014 19:01
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

  • Date - 07/16/2014 19:00 - 07/17/2014 11:53
  • Last Updated - 07/16/2014 19:00
Rebuild Apache/PHP (Resolved)
  • Priority - Medium
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.
    PHP is being updated to the newest minor version - no major version upgrades are being installed.

  • Date - 07/16/2014 19:00 - 07/17/2014 12:46
  • Last Updated - 07/16/2014 19:00
Rebuild Apache/PHP (Resolved)
  • Priority - High
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.

  • Date - 07/16/2014 16:53 - 07/16/2014 17:12
  • Last Updated - 07/16/2014 16:53
Rebuild Apache/PHP (Resolved)
  • Priority - High
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.

  • Date - 07/16/2014 16:53 - 07/16/2014 17:28
  • Last Updated - 07/16/2014 16:53
Rebuild Apache/PHP (Resolved)
  • Priority - High
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.

  • Date - 07/16/2014 15:14 - 07/16/2014 16:50
  • Last Updated - 07/16/2014 15:14
Rebuild Apache/PHP (Resolved)
  • Priority - High
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.

  • Date - 07/16/2014 15:14 - 07/16/2014 16:50
  • Last Updated - 07/16/2014 15:14
Rebuild Apache/PHP (Resolved)
  • Priority - High
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.

  • Date - 07/16/2014 14:59 - 07/16/2014 16:50
  • Last Updated - 07/16/2014 15:14
Rebuild Apache/PHP (Resolved)
  • Priority - High
  • An apache/PHP rebuild is required and in progress - for breif periods during the build you website may be unavailable.

  • Date - 07/16/2014 14:45 - 07/16/2014 14:55
  • Last Updated - 07/16/2014 14:45
Server not responding (Resolved)
  • Priority - High
  • This server is currently down - we are actively investigating the cause.

    ---

    Server is booting howerver a mandatory forced FSCK (disk check) is now in progress.  Usually takes approx 20 min

  • Date - 07/02/2014 10:10 - 07/02/2014 11:58
  • Last Updated - 07/02/2014 10:14
Server reboot in progress (Resolved)
  • Priority - High
  • We have had to restart this server and it is currently performing a quota operation mandatory in OpenVZ.  Once this operation completes, the server should come back online.
    ETA 20 min

  • Date - 06/22/2014 14:46 - 06/22/2014 15:43
  • Last Updated - 06/22/2014 14:47
Server listed at bl.spamcop.net (Resolved)
  • Priority - High
  • We have resolved the issue causing this listing and have requested delist from spamcop.net.

  • Date - 06/18/2014 11:51 - 06/22/2014 10:17
  • Last Updated - 06/18/2014 11:51
Queue flooded with spam (Resolved)
  • Priority - Medium
  • A hacked mail account has caused the mail queue to be flooded with spam.  We have identified and stopped the spammer and clearing the queue - at this time the server is under high load cleaning he affected queue.

  • Date - 06/14/2014 17:44 - 06/14/2014 22:30
  • Last Updated - 06/14/2014 17:45
Server mysql under high load (Resolved)
  • Priority - Medium
  • We are investigating a continuous load issue with this server - in this process we are shutting down mysql for approx 20min to see if the processes are to heavy for the server.

  • Date - 06/11/2014 21:42 - 06/14/2014 17:44
  • Last Updated - 06/11/2014 21:43
Server reboot in progress (Resolved)
  • Priority - High
  • We have had to restart this server because of load issue investigations.  Because of the restart, a mandatory disk check is now in progress.  We realize the timing on the forced disk check is during normal business hours however it is better to allow the disk check to complete at this time.  ETA 30 min.

  • Date - 06/11/2014 11:20 - 06/11/2014 21:42
  • Last Updated - 06/11/2014 11:21
Server not responding (Resolved)
  • Priority - High
  • We are investigating a server issue affecting multiple servers including this one.  We hope to have an update soon.

  • Date - 06/06/2014 02:00 - 06/06/2014 11:09
  • Last Updated - 06/10/2014 11:09
MySQL Upgrade / cPanel Upgrade (Resolved)
  • Priority - Medium
  • In order to update cpanel/WHM to the latest version, MySQL 5.5 is required.  MySQL 5.0 is deprecated and no longer supported.  As such, in order to bring this server in line with all others in our infrastructure we are in the process of updating MySQL.  During this time, websites requiring mysql queries may not be available.

  • Date - 06/01/2014 09:48 - 06/06/2014 08:40
  • Last Updated - 06/01/2014 09:50
Apache/PHP recompile (Resolved)
  • Priority - Medium
  • Due to some errors we are force to recompile Apache and PHP to the latest versions (2.2.x and 5.4.x respectively) - during this process your websites may become unavailable.

  • Date - 05/27/2014 10:43 - 05/27/2014 11:27
  • Last Updated - 05/27/2014 10:43
Queue flooded with spam (Resolved)
  • Priority - High
  • This server is experiencing slowdowns due to the mail queue being full of spam.  We have identified the offending sender and disabled their account.  We are clearing the queue and will take steps to mitigate this behaviour in the future.

  • Date - 05/05/2014 16:22 - 05/05/2014 21:52
  • Last Updated - 05/05/2014 16:23
cPanel Update (Resolved)
  • Priority - Medium
  • With a successful MySQL update completed, cPanel is being updated to the lastest RELEASE version 11.42.1.x

  • Date - 04/20/2014 11:56 - 05/04/2014 20:46
  • Last Updated - 04/20/2014 11:57
MySQL upgrade in progress (Resolved)
  • Priority - Medium
  • MySQL is being upgraded to 5.5 - during the upgrade process, websites which use databases will be unavailable

  • Date - 04/20/2014 11:19 - 04/20/2014 11:56
  • Last Updated - 04/20/2014 11:20
PHP update in progress (Resolved)
  • Priority - Medium
  • PHP/Apache is being recompiled to the latest versions

  • Date - 04/20/2014 11:08 - 04/20/2014 11:41
  • Last Updated - 04/20/2014 11:09
Server updates (Resolved)
  • Priority - Medium
  • This server is being updated - as such some sites are not responding as apache and PHP are being updated.

  • Date - 03/15/2014 21:35 - 03/15/2014 21:53
  • Last Updated - 03/15/2014 21:35
WHM/Cpanel Uprade to 11.40.11 (Resolved)
  • Priority - Medium
  • We are currently upgrading MySQL from 5.0.x to 5.5.x
    We are currently upgrading WHM/cPanel from 11.38.x (EOL) to 11.40.11 - current release
    You may see some interruption in web services while the system updates it's packages.

  • Date - 02/28/2014 20:00
  • Last Updated - 02/28/2014 21:10
Backup LAN GB (Resolved)
  • Priority - Medium
  • Affecting System - Backup Swich - TOR1 DC1
  • One of our backup switches used for backend communcations is down.  This will affect communiction between websites and databases and backup operations.
    We are investigating the issue and hope to have a resolution shortly.

  • Date - 02/03/2014 09:50 - 02/03/2014 13:58
  • Last Updated - 02/03/2014 09:51
PHP upgrade to 5.4 (Resolved)
  • Priority - Medium
  • We have now started upgrading / recompiling apache with PHP 5.4 on this server.
    Should be done shortly.
    Build complete - PHP is now 5.4.24

  • Date - 01/20/2014 20:09 - 01/20/2014 21:00
  • Last Updated - 01/20/2014 20:38
MS Information store hung (Resolved)
  • Priority - Critical
  • Affecting System - Hosted MS Exchange 2007
  • Customers of our hosted MS Exchange have noticed intermittent connectivity this morning - We are working to resolve this and have initiated a server restart.

    --UPDATE--
    An integrity check and fix is required on one of the information stores.
    This necessitates bringing the store offline during the check.
    During this time some mailboxes may be unavailable depending on which store your mailbox resides.
    We appreciate your patience during this check/fix operation.
    --

    --UPDATE--
    Please note that is is a long procedure.  We estimate the time to completion at 14:30
    Messages will be queued on other servers until the information store becomes available
    --

    --UPDATE--
    Test revealed further errors which are now being fixed through another procedure.  We esitmate another 4 hours before completion.
    Our apologies for the inconvenience
    --

    --UPDATE--
    Integrity check is taking much longer than anticipated and we may have to restore from backup as we cannot continue to keep the server offline.
    We will work to also restore the data from the restore point until this morning but keep customer up to date via this post.
    --

    --UPDATE / SUNDAY MORNING--
    We have added some additional disk arrays and will be doing a restore from backup shortly.  Unfortunately our DB fix did not complete successfully and so we must restore from backup.  We will work to recover data manually from the corrupt DB after the server is back online.
    Again, we thank you for your patience.
    --

    --UPDATE--
    Restore is in progress and completion is estimated in approx 3 hours
    --

  • Date - 12/14/2013 09:15
  • Last Updated - 12/16/2013 20:00
IMAP/POP connections failing (Resolved)
  • Priority - Critical
  • We are investigating an issue whereby all IMAP and POP accounts cannot login to retrieve mail

  • Date - 12/14/2013 09:35 - 12/14/2013 10:19
  • Last Updated - 12/14/2013 09:36
Server not responding - High Load (Resolved)
  • Priority - Critical
  • This server is currently under high load and may not respond to web/email requests.
    -Update
    We have found an excessive amount of messages in the mail queue and are investigating.

  • Date - 12/13/2013 12:38 - 12/13/2013 13:01
  • Last Updated - 12/13/2013 12:53
Server not responding - High Load (Resolved)
  • Priority - High
  • This server is currently under high load and may not respond to web/email requests.

  • Date - 12/13/2013 12:38 - 12/14/2013 09:16
  • Last Updated - 12/13/2013 12:38
Smartermail - Michigan (Resolved)
  • Priority - High
  • Affecting Server - VIPER-HELM-SH
  • Currently our MI email server webm1-MI.entirelydigital.com is at 100%CPU - likely due to a DDOS attack - we are currently investigating.

  • Date - 12/12/2013 12:54 - 12/13/2013 12:38
  • Last Updated - 12/12/2013 12:55
Mandatory Disk Check (Resolved)
  • Priority - High
  • We have had to restart this server and force a mandatory disk check to ensure integrity.  During this time, the server and it's services will be unavailable.
    We anticipate downtime of approx 1 hour.

  • Date - 12/11/2013 18:23 - 12/11/2013 21:47
  • Last Updated - 12/11/2013 21:47
Apache offline temporarily (Resolved)
  • Priority - Medium
  • We are recompling apache to address some web server issues.  It should be up shortly.

  • Date - 12/09/2013 14:49 - 12/14/2013 10:19
  • Last Updated - 12/09/2013 14:50
cpanel20.onlinemountain.com currently down (Resolved)
  • Priority - Critical
  • WHM/Cpanel host cpanel20.onlinemountain.com is currently not repsonding.  We have our NOC technician looking at the server and we are investigating the issue.
    We hope to have it up and running shortly.

    --UPDATE--
    Server is undergoing a forced fsck (disk check) which we cannot stop.  This is a mandatory check after a hard reset.  We will investigate the cause of the issue to ensure that the resource drain does not re-occur.
    --UPDATE--

    --UPDATE--
    Server is running - we are now performing system checks before bringing WHM/cpanel back online
    --UPDATE--

    --UPDATE--
    Server has finished checks and WHM/cPanel is now initializing
    --UPDATE--

  • Date - 12/02/2013 10:19
  • Last Updated - 12/02/2013 12:22
cPanel Webmail Auto-Responder issue (Resolved)
  • Priority - Low
  • Affecting System - cPanel Webmail
  • There is an issue with webmail on all cPanel servers.  cPanel 11.40 does not allow you to set custom start and stop date/times for your autoresponder.  No calendar drop down appears when you click on the date.  This is a known issue and will be fixed with the next cPanel update.

  • Date - 10/15/2013 16:18
  • Last Updated - 11/22/2013 14:35
System restart (Resolved)
  • Priority - High
  • ed-sh-cp1.entirelydigital.com is currently undergoing a file system check.  Once complete the server will be back onine.

  • Date - 10/26/2013 08:58 - 10/26/2013 13:38
  • Last Updated - 10/26/2013 08:59
Exchange 2007 System Down (Resolved)
  • Priority - Critical
  • Affecting System - Exchange 2007
  • We are experiencing an outage on our Exchange 2007 server - we are working as quickly as possible to resolve the issue.  Mailbox access, webmail access is affected.
    UPDATE - All hardware upgraded and issue is now resolved

  • Date - 10/01/2013 07:30 - 10/07/2013 00:00
  • Last Updated - 10/15/2013 16:18
SmarterMail MI (Resolved)
  • Priority - Medium
  • Affecting System - webmail MI webmail.neiko.net
  • We are upgrading all our SmarterMail Servers to enterprise 11.6.x

  • Date - 09/24/2013 21:55
  • Last Updated - 09/24/2013 22:03
Datacenter Maintenance (Resolved)
  • Priority - Medium
  • Affecting System - UPS Battery
  • EntirelyDigital will be replacing a single UPS Battery at the Toronto data center location on Tuesday May 14, 2013 from 16:01hrs to 17:30hrs. There is no expected service impact during this maintenance

  • Date - 05/14/2013 17:01 - 11/27/2013 10:14
  • Last Updated - 05/10/2013 16:09
IP Blacklisted (Resolved)
  • Priority - Critical

  • Due to an internal spammer/compromised account, 208.92.222.102 has been blacklisted.

    We are investigating the issue and will request immediate delist once the account has been disabled.

  • Date - 04/03/2013 10:13 - 04/11/2013 10:14
  • Last Updated - 04/03/2013 10:13
High load - SMTP/POP/IMAP unresponsive (Resolved)
  • Priority - High
  • Affecting System - cas1.entirelydigital.com/smtp75.entirelydigital.com

  • We are experiencing very high loads on this server and are investigating the cause.  SMTP/POP and IMAP connections will time out and server will appear unresponsive.

  • Date - 03/06/2013 14:11 - 03/08/2013 17:15
  • Last Updated - 03/06/2013 14:12
Drive Failure (Resolved)
  • Priority - Critical
  • ED-SH-CP1.entirelydigital.com experienced a drive failure a short time ago.  We are investigating and hope to have the server up and running shortly.

  • Date - 11/29/2012 11:25 - 03/04/2013 22:27
  • Last Updated - 11/29/2012 11:25
Telephone Support Unavailable (Resolved)
  • Priority - Critical
  • Affecting System - Telephone Technical Support
  • We are currently experiencing issues with our telephone service. As you may or may not know, our phone servers are located in our US office. Last night a severe storm knocked out power, Internet, and essential services to almost a million homes in Northern Virginia. Luckily our US staff was unharmed by the storm. As of right now, the Gov. Bob McDonnell has declared a state of emergency for Virginia.

    Our vendors providing the DIDs and 'dial tone' for all of our phone lines have been forced to shut down all power, including UPS, so emergency crews can work to repair damaged power lines safely.

    As inconvenient and unfortunate as this is, we are doing everything we can to assist/comply with emergency crews and services in hopes of having this resolved soon.

    This telephony outage has no impact on helpdesk support services we provide.

    We do not have a firm ETA however we will keep you updated as soon as we get additional information from the upstream providers and/or Virginia Dominion Power.

    You can reach our support staff as usual by emailing helpdesk@entirelydigital.com

    We thank you for your patience and understanding. Our thoughts are with those who lost loved ones in last nights storm.

  • Date - 07/01/2012 00:00 - 07/01/2012 18:22
  • Last Updated - 07/01/2012 00:54
Network Outage - USA Datacenter (Resolved)
  • Priority - Critical
  • Affecting Other - Network Outage - USA Datacenter
  • We are experiencing a network outage in our USA datacenter - this is not a server issue but an issue with a downstream provider

    We are currently invesitgating and this issue is priority 1 - Critical

    -- update this issue is now resolved and we are actively monitoring the hardware in question

  • Date - 06/29/2012 15:15 - 06/29/2012 15:35
  • Last Updated - 06/29/2012 15:56
Michigan Data Center Outage (Resolved)
  • Priority - Critical
  • Affecting System - Michigan Data Center - some servers
  • We are experiencing a major outage in our Michigan data center and are actively investigating.

    Please do not email our helpdesk regarding your site or email being down as we are already aware of the issue and it is a network issue not a server issue or an issue with your particular domain.

  • Date - 06/21/2012 08:07 - 06/21/2012 10:05
  • Last Updated - 06/21/2012 08:14
US Datacenter Outage (Resolved)
  • Priority - Critical
  • Affecting Other - US Datacenter
  • We are experiencing a system wide datacenter outage in our USA datacenter.

    An investigation is currently in progress.  We are aware of the issue so please do not call our support line or email our helpdesk as your website/service/email will come back on line automatically once the issue is resolved.

  • Date - 06/03/2012 15:41 - 06/03/2012 00:00
  • Last Updated - 06/03/2012 16:54
Server listed at senderbase.org (Resolved)
  • Priority - Medium
  • We are currently investigating why this server (208.92.222.102) is listed at senderbase.org. This may affect you sending email to certain domains that use Cisco firewalls as their antispam solution.

  • Date - 04/30/2012 12:38 - 06/03/2012 16:54
  • Last Updated - 04/30/2012 12:39
Kernel security, bug fixes, and enhancement u (Resolved)
  • Priority - Medium
  • Affecting System - CPanel Servers
  • We will be scheduling an important kernel security, bug fixes and enhancement update for all our CentOS 5.x servers this weekend.

    This would take place as far as possible during the off-peak hours. A reboot is required to complete the upgrade. The downtime should not exceed 30 minutes and it will be minimize as much as possible. We will track each server till it returns to service after the upgrade and reboot.

    This upgrade is scheduled as follows:
    Date: 21 April 2012 (Saturday) to 23 April 2012 (Monday)
    Time: Between 2AM and 8AM EST

    Details of Kernel security, bug fixes and enhancement update
     
    * A flaw in the xfrm6_tunnel_rcv() function in the Linux kernel's IPv6 implementation could lead to a use-after-free or double free flaw in tunnel6_rcv(). A remote attacker could use this flaw to send
    specially-crafted packets to a target system that is using IPv6 and also has the xfrm6_tunnel kernel module loaded, causing it to crash. (CVE-2012-1583, Important)

    If you do not run applications that use xfrm6_tunnel, you can prevent the xfrm6_tunnel module from being loaded by creating (as the root user) a "/etc/modprobe.d/xfrm6_tunnel.conf" file, and adding the following line to it:

    blacklist xfrm6_tunnel

    This way, the xfrm6_tunnel module cannot be loaded accidentally. A reboot is not necessary for this change to take effect.

    This update also fixes various bugs and adds an enhancement.

  • Date - 04/20/2012 02:00 - 07/01/2012 00:55
  • Last Updated - 04/20/2012 15:38
HSphere Server Migration (Resolved)
  • Priority - Medium
  • We are in the process of migrating the Hsphere control panel server. During this move, access to account functions will not be available.  Web and email services are unaffected.

  • Date - 04/10/2012 19:57 - 04/12/2012 11:42
  • Last Updated - 04/10/2012 19:58
Michigan IDC - Unscheduled Maintenance (Resolved)
  • Priority - High
  • Affecting System - Michigan IDC - All Services
  • We are currently performing some unscheduled maintenance on our Michigan SAN infrastructure.  We estimate service will resume at approx 4am EST

  • Date - 01/13/2012 01:14 - 01/13/2012 12:01
  • Last Updated - 01/13/2012 01:15
Email services unavailable (Resolved)
  • Priority - High
  • Affecting System - some email servers
  • One of our email servers is currently not accepting requests for POP/IMAP - service will be restored shortly.

  • Date - 12/06/2011 09:35 - 12/06/2011 10:21
  • Last Updated - 12/06/2011 09:36
Cpanel Licence Error (Resolved)
  • Priority - Medium
  • CP3.shieldhost.com is currently showing an error.  This affects webmail and control panel access.  We are working with the control panel vendor to resolve this.

  • Date - 11/14/2011 10:00 - 11/14/2011 11:34
  • Last Updated - 11/14/2011 10:01
Unable to access plesk / download mail (Resolved)
  • Priority - Critical
  • Affecting Server - NET-209
  • Clients trying to access POP and IMAP mail will not be able to login and will recieve an error in their mail clients as well as webmail.

    We are aware of the issue and working toward a resolution.

  • Date - 10/14/2011 16:01 - 10/17/2011 14:17
  • Last Updated - 10/14/2011 16:20
Worldwide BES issues (Resolved)
  • Priority - High
  • Affecting System - Hosted Exchange 2007 / BES - Blackberry Enterprise Server
  • Blackberry's (RIM) outage has spread to Canada and affects all BES users on hosted MS exchange. This is not an issue with EntirelyDigital's networks but stems from RIM itself.

    There is no ETA for resolution.

    iPhone, Android, Windows Phone users are unaffected by this outage.

  • Date - 10/12/2011 07:33 - 10/17/2011 14:17
  • Last Updated - 10/12/2011 09:39
Hosted Exchange 2007 - Maintenance (Resolved)
  • Priority - Medium
  • Affecting Other - Hosted Exchange 2007
  • Dear Microsoft Exchange User, (french message follows)
    On Sunday March 27th, 2011 at approximately 22:00 (10pm) EST (GMT -5.00) we will be doing some software and hardware upgrades on our Microsoft Exchange infrastructure.
    During this time your mailbox will be unavailable. The duration of this maintenance window is estimated to be approximately 1 hour.
    With these upgrades we will be increasing our capacity which means larger mailbox quotas coming soon.
    This upgrade is part of many infrastructure upgrades we will be doing in 2011.
    Regards,
    EntirelyDigital NOC Team

    ---

    Cher utilisateur de Microsoft Exchange,

    Le 27 Mars 2011, à environ 22h00 EST (GMT -5.00), nous allons faire quelques mises à jour logicielles et matérielles de notre infrastructure Microsoft Exchange.
    Pendant ce temps, votre boîte aux lettres ne seront pas disponibles. La durée de cette fenêtre de maintenance est estimée à environ 1 heure.

    Avec ces améliorations, nous augmenterons notre capacité ce qui signifie que les quotas de boîtes aux lettres plus à venir.

    Cette mise à jour fait partie de nombreuses améliorations à l'infrastructure que nous allons faire en 2011.

    Cordialement,

    EntirelyDigital équipe CNP

  • Date - 03/27/2011 22:00 - 04/22/2011 20:12
  • Last Updated - 03/26/2011 19:29
Network Upgrades (Resolved)
  • Priority - Low
  • Affecting Other - Michigan IDC
  • EntirelyDigital is scheduling a service window from Tuesday 1/18/2011 11:00PM
    to Wednesday 1/19/2011 2:00AM to perform upgrades on key network equipment in our Michigan Data Centre. We
    do not expect any network outages durring this service window. Customers may notice
    a temporary changes in routing.

    Level (3) connectivity will temporarily shutdown durring the service widow. All
    customer traffic will route through other peers during this time.

    This scheduled maintenance is part of EntirelyDigital's constant commitment
    to providing customers with the very best in Internet service.

  • Date - 01/18/2011 00:00 - 01/26/2011 20:42
  • Last Updated - 01/26/2011 20:42
Exchange server unavailable for some users (Resolved)
  • Priority - High
  • Affecting System - Exchange 2007
  • We are moving some mailboxes to a new storage array and as such they may be unavailable for approx 15 minutes.

    Your connection will become available once the operation is complete.

  • Date - 01/14/2011 20:49 - 01/14/2011 22:00
  • Last Updated - 01/18/2011 09:50
Hosted Exchange Temporarily Unavailable (Resolved)
  • Priority - Critical
  • Affecting Other - Postoffice.entirelydigital.com
  • We had a backup run longer than expected and have cut it short this morning. As a preventative measure we have restarted the server. All exchange services will be back online in approximately 5 minutes. All messages queued at the Edge servers will be delivered as soon as it comes online.

    Sorry for any inconvenience.

    NOC Team

  • Date - 01/13/2011 07:52 - 01/13/2011 11:12
  • Last Updated - 01/13/2011 07:54
Mail servers updates (Resolved)
  • Priority - Low
  • Affecting Other - Updates
  • All mail servers in the EntirelyDigital network will be updated to their latest build starting 22:00 EST. Minor interruptions in mail flow and mailbox access expected during the upgrades.

  • Date - 10/12/2020 15:22 - 12/21/2010 22:57
  • Last Updated - 12/15/2010 17:07
Hosted Exchange Down (Resolved)
  • Priority - Critical
  • Affecting System - Hosted Exchange Customers - All
  • We are experiencing hosted Exchange Connectivity issues.

    Currently all messages are flowing correctly, however we will be doing a full database consistency check at approximately 10:00PM EST

  • Date - 12/04/2010 22:22 - 12/10/2010 10:50
  • Last Updated - 12/05/2010 12:32
Email server down sporatically (Resolved)
  • Priority - Critical
  • Affecting System - Email - Toronto Data Centre
  • Email services were temporarily down between 16:00 EST and 16:30 EST today. We have resolved the issue and the server is operating normally.

  • Date - 11/04/2010 16:00 - 11/04/2010 16:23
  • Last Updated - 11/04/2010 16:24
Michigan Data Centre Network Issue (Resolved)
  • Priority - Critical
  • We are experiencing connectivity issues with some of our core routers in our Michigan Data Centre.

    We are investigating the proplem and hope to have a resolution shortly.

  • Date - 09/16/2010 11:19 - 09/16/2010 11:28
  • Last Updated - 09/16/2010 11:20
Server Down (Resolved)
  • Priority - Critical
  • Affecting Server - VIPER-DNP01-SH
  • We are experiencing an outage with some of our customers websites hosted through the Michigan data centre.

    Our NOC team is aware of the issue and is actively working on the problem.

  • Date - 08/26/2010 07:58 - 08/26/2010 11:34
  • Last Updated - 09/09/2010 17:34
Some webmail clients may not see all messages (Resolved)
  • Priority - Medium
  • In some cases once you logon to your webmail client, you may not see your messages or you may only see a partial list.

    If this is the case your webmail client needs to be reset and messages re-indexed by the application.  Should this be the case, please submit a helpdesk ticket and logout of your webmail.  We will re-initialize your client and advise once you can logon again.

  • Date - 06/28/2010 00:00 - 08/01/2010 00:25
  • Last Updated - 06/28/2010 23:54
Upgrade to Offsite Backup Server and Software (Resolved)
  • Priority - Medium
  • Affecting Server - OBS
  • We will be upgrading our backup server to the latest build 5.5.7.  All installers will also be updated and your autoupdate utility should update your backup software to this version.  You may receive old failure notifications as we purge some of our logs.

  • Date - 06/03/2010 13:00 - 06/28/2010 23:52
  • Last Updated - 06/03/2010 10:19
Updates to hosted exchange infrastructure (Resolved)
  • Priority - Critical
  • Affecting System - Hosted Exchange 2007
  • We will be performing updates to our hosted exchange infrastrucutre.  Mailbox access will be temporarily unavailable at various points during the update.

    Mail will be queued at the edge transport level so no messages will be lost.

  • Date - 06/02/2010 19:00 - 06/03/2010 10:19
  • Last Updated - 06/03/2010 10:19
Updates to Viperhost DNP (Resolved)
  • Priority - Medium
  • Affecting Server - VIPER-DNP01-SH
  • We will be updating the control panel software which connects/configures all Viperhost services over the weekend.  You may experience periods where your website / email may not be operational for brief periods (less than 5 minutes at a time) while we restart services.

  • Date - 04/23/2010 21:00 - 04/25/2010 21:00
  • Last Updated - 04/29/2010 13:49
Updates to Hosted Exchange Server (Resolved)
  • Priority - Medium
  • Affecting System - Hosted Exchange
  • Between April 23rd 9:00pm EST and April 25th 9:00pm EST we will be performing maitenance on our Hosted Exchange infrastructure.  You may experience connectivity issues during the weekend only for a few minutes while we restart services.

  • Date - 04/23/2010 21:00 - 04/29/2010 13:49
  • Last Updated - 04/22/2010 21:33
Viperhost Michigan Data Centre (Resolved)
  • Priority - Critical
  • Affecting System - Viperhost Michigan Data Centre
  • EntirelyDigital experienced a failure of a core router in Southfield. Some of
    the fail-over mechanisms did not work, requiring manual innervation. This resulted
    in an interruption to some customers in the Viperhost Michigan data center. Currently service
    has been restored and was unavailable for approximately 9 minutes

  • Date - 04/20/2010 14:52 - 04/20/2010 15:00
  • Last Updated - 04/20/2010 16:31
Scheduled Network Upgrades (Resolved)
  • Priority - Medium
  • Affecting System - Michigan Data Centre
  • Affecting:
    Customers in Viperhost Michigan Data Center

    Details:
    EntirelyDigital Viperhost is scheduling a service window from 7:00PM-10:00PM Friday, April
    9th, 2010, to perform upgrades on key network equipment.

    Level (3) connectivity will temporarily shutdown for a period of not more then
    approximately 15 minutes. All customer traffic will route through other peers during
    this time.  As such no downtime is expected

  • Date - 04/09/2010 00:00 - 04/09/2010 00:00
  • Last Updated - 04/13/2010 20:46