Using google business and drive to automate db backup

In this example, I create a Windows service that manages Google Drive.  The service calls a batch file to create an Oracle backup using EXP, manages space on Google Drive and then uploads the backup to google, finally sending an html email via smtp.

The school of hard knocks is a great teacher.  Having a hosting provide fail can be a nightmare.  Invariably this happens without notice and suddenly you are in scramble mode and without sleep for days moving.  There are ways to hedge the dependency on your hosting provider.   One way is to host your DNS with a third party.  This way should things go sideways, you can readily change your DNS settings.  Another good practice is to keep a scheduled cold backup of your database independent of your hosting provider.  Google Drive is a great choice for this.  Not only has google made a commitment to putting their servers up on 10gbs, if your servers are on 1gbs or 10gbs copying large files to google is smoking fast.  The db using the code in this example creates an 800gb cold backup, it takes under 15 seconds to put up on google drive.

Automating this backup process with google drive and Google Business is a bit convoluted, there are security issues and settings that are simpler with a gmail account, or no associated account.  Using Google Business to host your transnational mail is a smart move.  No one messes with Google for mail delivery into the inbox.  Why struggle with a shared ip or building an ip reputation for transnational mail, marketing mail is a different story, but for transnational mail there is no better choice then Google for Business for a small company.  Drive allocation on Google Business, per user, is 30gb vs 15 with gmail.

This examples uses the Google REST API v3, Visual Studio 2012 in C#
Reference https://developers.google.com/drive/v3/web/about-sdk

In a user scenario Google authenticates via a challenge via the browser, unfortunately, that will not work here because we are creating a Service Application, in this case a Windows Service.  Services do not have access to an active user session, fortunately, Google anticipated this need and supports offline authentication for service accounts.

Setup a Service Account on Google
Access console.developers.google.com using admin credentials for your business account.
Locate (the every changing) section for Google Apps APIs
Select Drive (formerly google docs)
Enable Google drive from the API Manager.
Select Credentials (left pane)
Create Credentials as a Service Account key using a project name, this project uses "ServerBackup" but you can name it whatever you want just set in the config.
Under permissions, assign the user name user@yourdomain Permission to use "ServerBackups"  this is how google knows which user account for drive.  From this step you need the internal service-account google created, this is in the form of an email address, it is not the same as a user email address, which is assigned permission to use the service account.  These values go in the config file. This is the connection between the internal service account and the user account.  It is possible to omit the user account in your code when you create the credentials, the service account is created with 15gb of storage, but there is no way to access the service account other than the api.  This is the only way I could this to work was without the user name until I figured out how to associate a user name to the service account.   If you do not care, nor need to access what you are uploading via drive.google.com and 15gb is fine, skip it.  The final step is to create the p12 key for the service account.  Under credentials, select the service account, then manage service accounts, then select create key and download in p12 format.  This is the private key, handle as you wish.  Because google is forever changing their interfaces, screen shots have the lifespan of a fruit-fly.

To implement, this example creates a basic windows service which calls an assembly.

This example only shows how to create the assembly using VS 2012 and 4.0 of the assembly for x64.

Create a new class library and open the Package Manager Console.
PM>Install-Package Google.Apis.Drive.v3
(only do this for the assembly. not the service exe)

Below is the code:

using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using System.Diagnostics;
using System.Configuration;
using System.IO;
using System.Data;
using System.Net;
using System.Net.Mail;
using System.Threading;
using System.Security.Cryptography.X509Certificates;
using Google.Apis.Auth.OAuth2;
using Google.Apis.Drive.v3;
using Google.Apis.Drive.v3.Data;
using Google.Apis.Services;
using Google.Apis.Util.Store;
 
namespace GPHousekeepingServer
{
 
    public class SMTPRec
    {
        public bool processConfirmations;
        public string host;
        public string hostUserName;
        public string hostPassword;
        public int hostPort;
        public string to;
        public string from;
        public string htmlSource;
        public string rawHTML;
        public string bcc;
        public string subject;
        public bool useSSL;
        public SMTPRec(string passedhtml, string passedSubject)
        {
            try
            {
                //because this is running from an assembly, cannot use ConfigurationManager directly.
                useSSL = false;
                string path = System.Reflection.Assembly.GetExecutingAssembly().Location;
                Configuration config = ConfigurationManager.OpenExeConfiguration(path);
 
                host = config.AppSettings.Settings["SmtpHost"].Value;
                string sPort = config.AppSettings.Settings["SmtpPort"].Value;
                if (sPort.Equals(string.Empty))
                    throw new Exception("SmtpPort is null");
                if (!int.TryParse(sPort, out hostPort))
                    throw new Exception("Unable to convert SmtpPort to an int.");
 
                if (config.AppSettings.Settings["SmtpHostUserName"] != null)
                {
                    hostUserName = config.AppSettings.Settings["SmtpHostUserName"].Value;
                    if (config.AppSettings.Settings["SmtpHostPassword"] == null)
                        throw new Exception("SmtpHostUserName set, however SmtpHostPassword is null");
                    hostPassword = config.AppSettings.Settings["SmtpHostPassword"].Value;
                }
 
                if (config.AppSettings.Settings["SmtpUseSSL"] != null)
                    useSSL = config.AppSettings.Settings["SmtpUseSSL"].Value.Equals("1");
 
                to = config.AppSettings.Settings["ToEmail"].Value;
                from = config.AppSettings.Settings["FromEmail"].Value;
                bcc = string.Empty;
 
                if (passedSubject.Equals(string.Empty))
                    subject = config.AppSettings.Settings["subject"].Value;
                else
                    subject = passedSubject;
 
                rawHTML = passedhtml;
 
            }
            catch (Exception E)
            {
                throw new Exception("Exception raised in SMTPRec constructor, " + E.Message);
            }
        }
    }
 
    public static class HouseKeeping
    {
 
        private static CollateralClass c = null;
        private static Thread thread = null;
 
        public static void Start()
        {
            try
            {
                if (c != null)
                    throw new Exception("Thread is already running.");
 
                c = new CollateralClass();
                c.bTerminated = false;
 
                thread = new Thread(new ThreadStart(c.Worker));
                thread.Start();
 
            }
            catch (Exception E)
            {
                Utility.SendEventLogError("Fatal error in Housekeeping (stopping), " + E.Message);
            }
        }
 
        public static void Stop()
        {
            try
            {
 
                if (c == null)
                    throw new Exception("Thread is not running, call Execute to start thread.");
                c.bTerminated = true;
 
            }
            catch (Exception E)
            {
                Utility.SendEventLogError("Exception raised in Stop, " + E.Message);
            }
        }
 
 
        class CollateralClass
        {
            public bool bTerminated = false;
            EventLog eventLog = new EventLog();
            Hashtable htReported = new Hashtable();
            public void Worker()
            {
 
                int iLongerJobCounter = 361;
                try
                {
                    while (!bTerminated)
                    {
                        Thread.Sleep(1000 * 5);
                        iLongerJobCounter++;
                        //runs every five minutes
                        if (iLongerJobCounter > 60)
                        {
                            iLongerJobCounter = 0;
                            RunDBBackupJob();
                        }
                    }
 
                }
                catch (Exception E)
                {
                    Utility.SendEventLogError("Exception raised in HousekeepingsServer (thread), " + E.Message);
                }
            }
        }
 
 
        private static void RunDBBackupJob()
        {
            try
            {
 
                DateTime dtLastTime = new DateTime(Utility.GetLastBackup());
                bool bBackup = (DateTime.Now > dtLastTime.AddMinutes(Utility.GetBackupEvery()));
 
                if (bBackup)
                {
                    string sFileName = Utility.GetBackupPrefix()+ DateTime.Now.ToString("yyyyMMdd_HHmm");
                    string sPathName = Utility.GetBackupLocation();
                    Utility.SendEventLogInfo("GPHousekeepingServer:RunDBBackUpJob to "+sPathName+sFileName);
 
                    var proc = new Process
                    {
                        StartInfo =
                            new ProcessStartInfo
                            {
                                FileName = Utility.GetBackupBatchFile(),   
                                Arguments = sPathName+sFileName,
                                UseShellExecute = false,
                                CreateNoWindow = false
                            }
                    };
                    proc.Start();
 
                    proc.WaitForExit();
                    int successExitCode = 0;
 
                    if (proc.ExitCode == successExitCode)
                    {
                        Utility.SendEventLogInfo("GPHousekeepingServer:RunDBBackUpJob done with exp, setting up drive");
 
                        var certificate = new X509Certificate2(Utility.GetBackupP12File(), "notasecret", X509KeyStorageFlags.Exportable);
                        var serviceAccountEmail = Utility.GetBackupServiceAccount();
                        var userAccountEmail = Utility.GetBackupUserAccount();
                        ServiceAccountCredential credential = new ServiceAccountCredential(
                            new ServiceAccountCredential.Initializer(serviceAccountEmail)
                            {
                                Scopes = new[] { Google.Apis.Drive.v3.DriveService.Scope.Drive },
                                User = userAccountEmail
                            }.FromCertificate(certificate));
 
                        // Create the service.
                        using (DriveService driveService = new DriveService(new BaseClientService.Initializer()
                            {
                                HttpClientInitializer = credential,
                                ApplicationName = Utility.GetApplicationName()
                            }))
                        {
                            var about = driveService.About.Get();
 
                            about.Fields = "storageQuota,user";
                            User user = about.Execute().User;
                            StringBuilder sb = new StringBuilder("<!DOCTYPE><html><head><title></title></head><body>");
                            sb.Append("User " + user.DisplayName + ", " + user.EmailAddress + "<br/>");
 
                            if (!System.IO.File.Exists(sPathName + sFileName + ".dmp"))
                                throw new Exception("Unable to location: " + sPathName + sFileName + ".dmp, did " + Utility.GetBackupBatchFile() + " run?");
 
                            FileInfo fi = new FileInfo(sPathName + sFileName + ".dmp");
 
                            //upload to drive.google
                            var storageQuota = about.Execute().StorageQuota;
                            if (storageQuota.Limit == null)
                                sb.Append("StorageUsage: " + Math.Round((double)(storageQuota.Usage / Math.Pow(1024, 2)), 1).ToString() + "mb, StorageLimit: Unlimited<br/>");
                            else
                            {
                                sb.Append("StorageUsage: " + Math.Round((double)(storageQuota.Usage / Math.Pow(1024, 2)), 1).ToString() + "mb, StorageLimit: " + (storageQuota.Limit / Math.Pow(1024, 3)) + "gb, Space Available: " + Math.Round((double)((storageQuota.Limit - storageQuota.Usage) / Math.Pow(1024, 2)), 1).ToString() + "mb<br/>");
                                if (fi.Length > (storageQuota.Limit - storageQuota.Usage))
                                {
 
                                    //clean up some space by removing the oldest files.  need to remove fi.length
                                    sb.Append("drive.google is full, making space by removing:<br/>");
 
                                    FilesResource.ListRequest listDelete = driveService.Files.List();
                                    // listRequest.Spaces = "appDataFolder";
                                    listDelete.PageSize = 100;
                                    listDelete.Fields = "nextPageToken, files(id, name, size, createdTime)";
                                    listDelete.OrderBy = "createdTime asc";
                                    IList<Google.Apis.Drive.v3.Data.File> files = listDelete.Execute().Files;
 
                                    long fucking=(long)storageQuota.Limit;
                                    long stupid = (long)storageQuota.Usage;
                                    long lBytesDeleted = fucking-stupid;
                                    if (files != null)
                                    {
                                        foreach (var file in files)
                                        {
                                            if (file.Name.ToUpper().Contains("BBB_"))
                                            {
                                                sb.Append("Deleted: " + ((DateTime)file.CreatedTime).ToString("MM/dd/yyyy HHmm") + " " + file.Name + " " + file.Id + " size: " + file.Size + "<br/>");
                                                driveService.Files.Delete(file.Id).Execute();
                                                lBytesDeleted += (long)file.Size;
                                                if (lBytesDeleted > fi.Length)
                                                    break;
                                            }
                                        }
                                    }
                                }
                            }
 
                            //upload file
                            Utility.SendEventLogInfo("GPHousekeepingServer:RunDBBackUpJob " + sFileName + " uploading to drive");
                            Google.Apis.Drive.v3.Data.File fileMetaData = new Google.Apis.Drive.v3.Data.File();
                            fileMetaData.Name = sFileName + ".dmp";
                            using (FileStream uploadStream = new FileStream(sPathName + sFileName + ".dmp", FileMode.Open, FileAccess.Read))
                            {
                                FilesResource.CreateMediaUpload request = driveService.Files.Create(fileMetaData, uploadStream, "application/octet-stream");
                                request.Fields = "id,size";
 
                                request.Upload();
                                uploadStream.Close();
                                uploadStream.Dispose();
 
                                Utility.SendEventLogInfo("GPHousekeepingServer:RunDBBackUpJob File Uploaded to drive, ID: " + request.ResponseBody.Id + ", size: " + request.ResponseBody.Size + "<br/>");
                                sb.Append("File Uploaded, ID: " + request.ResponseBody.Id + ", size: " + request.ResponseBody.Size + "<br/>");
 
                            }
                            {
                                FilesResource.ListRequest listRequest = driveService.Files.List();
                                // listRequest.Spaces = "appDataFolder";
                                listRequest.PageSize = 100;
                                listRequest.Fields = "nextPageToken, files(id, name, size, createdTime)";
                                listRequest.OrderBy = "createdTime desc";
                                IList<Google.Apis.Drive.v3.Data.File> files = listRequest.Execute().Files;
 
                                if (files != null)
                                {
                                    sb.Append("FILES on drive.google.com<br/>");
                                    foreach (var file in files)
                                        sb.Append(((DateTime)file.CreatedTime).ToString("MM/dd/yyyy HHmm") + " " + file.Name + " " + file.Id + " size: " + file.Size + "<br/>");
                                }
                            }
                            Utility.SetLastBackup(DateTime.Now.Ticks);
                            sb.Append("</body></html>");
 
                            SendEMail(new SMTPRec(sb.ToString(), "dbBackup"));
                            driveService.Dispose();
                        }
                    }
                    else
                    {
                        throw new Exception(string.Format("EXP FAILED!  Process exit code not same as successExitCode specified.!! Process ExitCode={0} and successExitCode={1}", proc.ExitCode, successExitCode));
                    }
                }
            }
            catch (Exception E)
            {
                Utility.SendEventLogError("Fatal Exception raised in HousekeepingServer.RunDBBackupJob (not-stopping), " + E.Message);
            }
        }
 

        private static void SendEMail(SMTPRec smtpInfo)
        {
            try
            {
                string sBody = smtpInfo.rawHTML;
 
                MailAddress maFrom = new MailAddress(smtpInfo.from, "Housekeeping", Encoding.UTF8);
                MailAddress maTo = new MailAddress(smtpInfo.to, smtpInfo.to, Encoding.UTF8);
 
                MailMessage mailMessage = new MailMessage(maFrom, maTo);
                if (!smtpInfo.bcc.Equals(string.Empty))
                    mailMessage.Bcc.Add(smtpInfo.bcc);
                mailMessage.Body = sBody;
                mailMessage.BodyEncoding = Encoding.UTF8;
                mailMessage.IsBodyHtml = true;
 
                mailMessage.Subject = smtpInfo.subject;
                mailMessage.SubjectEncoding = Encoding.UTF8;
 
                SmtpClient smtpClient = new SmtpClient(smtpInfo.host, smtpInfo.hostPort);
                smtpClient.DeliveryMethod = SmtpDeliveryMethod.Network;
                smtpClient.Timeout = 20000;
                if (smtpInfo.useSSL)
                {
                    Utility.SendEventLogInfo("Using SSL");
                    smtpClient.EnableSsl = true;
                }
 
                if (!string.IsNullOrEmpty(smtpInfo.hostUserName))
                {
                    Utility.SendEventLogInfo("Setting credentials smtpClient, " + smtpInfo.hostUserName + " password " + smtpInfo.hostPassword);
                    smtpClient.UseDefaultCredentials = false;
                    smtpClient.Credentials = new NetworkCredential(smtpInfo.hostUserName, smtpInfo.hostPassword); ;
                }
                smtpClient.Send(mailMessage);
            }
            catch (Exception E)
            {
                Utility.SendEventLogError("GPHousekeeping:SendEMail Exception Raised: "+ E.Message);
            }
        }
 
    }
 }

Utility.cs

using System;
using System.Data;
using System.Configuration;
using System.Text.RegularExpressions;
using System.IO;
using System.IO.Ports;
using System.Net;
using System.Net.Sockets;
using Oracle.DataAccess.Client;
using System.Diagnostics;
 
 
/// <summary>
/// Summary description for Utility
/// </summary>
public static class Utility
{
    public static System.Diagnostics.EventLog myEventLog;
 
    static Utility()
    {
        if (!EventLog.SourceExists("GPHousekeepingServer"))
            EventLog.CreateEventSource("GPHousekeepingServer", "Application");
        myEventLog = new EventLog("Application");
        myEventLog.Source = "GPHousekeepingServer";
    }
 
    public static void SendEventLogError(string errorMessage)
    {
        try
        {
            if (myEventLog != null)
                myEventLog.WriteEntry(errorMessage, EventLogEntryType.Error);
        }
        catch (Exception)
        {
            //nothing to do with this... :(
        }
    }
 
    public static void SendEventLogInfo(string infoMessage)
    {
        try
        {
            if (myEventLog != null)
                myEventLog.WriteEntry(infoMessage, EventLogEntryType.Information);
        }
        catch (Exception)
        {
            //nothing to do with this... :(
        }
    }
 
    public static bool isEmail(string inputEmail)
    {
        string strRegex = @"^([a-zA-Z0-9_\-\.]+)@((\[[0-9]{1,3}" +
              @"\.[0-9]{1,3}\.[0-9]{1,3}\.)|(([a-zA-Z0-9\-]+\" +
              @".)+))([a-zA-Z]{2,4}|[0-9]{1,3})(\]?)$";
        Regex re = new Regex(strRegex);
        if (re.IsMatch(inputEmail))
            return (true);
        else
            return (false);
    }
 
 
    public static uint IPDottedToUint(string dottedIP)
    {
        uint uIP = 0;
        try
        {
            string[] sNumbers = dottedIP.Split('.');
            if (sNumbers.GetLength(0) != 4)
                throw new Exception("invalid format, " + dottedIP);
 
            uIP = (uint.Parse(sNumbers[0]) * 16777216) + (uint.Parse(sNumbers[1]) * 65536) + (uint.Parse(sNumbers[2]) * 256) + (uint.Parse(sNumbers[3]));
 
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in IPDottedToUint, " + E.Message);
        }
        return uIP;
    }
 
 
    public static long GetLastBackup()
    {
        long lResult=0;
        try
        {
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["lastBackupRun"] == null)
            {
                //to force initial run, arbitrarily subtract one day
                lResult = DateTime.Now.AddDays(-1).Ticks;
            }
            else
            {
                if (!long.TryParse(config.AppSettings.Settings["lastBackupRun"].Value, out lResult))
                    throw new Exception("Trouble parsing lastBackupRun (ticks) from config, " + config.AppSettings.Settings["lastBackupRun"].Value);
            }
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in GetLastBackup, " + E.Message);
        }
        return lResult;
    }
 
    public static void SetLastBackup(long ticks)
    {
        try
        {
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["lastBackupRun"] == null)
                config.AppSettings.Settings.Add("lastBackupRun", ticks.ToString());
            else
                config.AppSettings.Settings["lastBackupRun"].Value = ticks.ToString();
            config.Save(ConfigurationSaveMode.Minimal, false);
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in SetLastBackup, " + E.Message);
        }
    }
 
    public static string GetBackupLocation()
    {
        string sResult = string.Empty;
        try
        {
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["backuplocation"] == null)
                throw new Exception("backuplocation is not set in App.config");
            sResult = config.AppSettings.Settings["backuplocation"].Value;
 
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in GetBackupLocation, " + E.Message);
        }
        return sResult;
    }
 
    public static string GetBackupServiceAccount()
    {
        string sResult = string.Empty;
        try
        {
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["backupserviceAccount"] == null)
                throw new Exception("backupserviceAccount is not set in App.config");
            sResult = config.AppSettings.Settings["backupserviceaccount"].Value;
 
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in GetBackupServiceAccount, " + E.Message);
        }
        return sResult;
    }
 
    public static string GetBackupUserAccount()
    {
        string sResult = string.Empty;
        try
        {
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["backupuseraccount"] == null)
                throw new Exception("backupuseraccount is not set in App.config");
            sResult = config.AppSettings.Settings["backupuseraccount"].Value;
 
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in GetBackupUserAccount, " + E.Message);
        }
        return sResult;
    }
 
    public static string GetBackupPrefix()
    {
        string sResult = string.Empty;
        try
        {
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["backupprefix"] == null)
                throw new Exception("backupprefix is not set in App.config");
            sResult = config.AppSettings.Settings["backupprefix"].Value;
 
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in GetBackupPrefix, " + E.Message);
        }
        return sResult;
    }
 
 
    public static string GetApplicationName()
    {
        string sResult = string.Empty;
        try
        {
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["backupapplicationname"] == null)
                throw new Exception("backupapplicationname is not set in App.config");
            sResult = config.AppSettings.Settings["backupapplicationname"].Value;
 
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in GetApplicationName, " + E.Message);
        }
        return sResult;
    }
 
 
 
    public static string GetBackupBatchFile()
    {
        string sResult = string.Empty;
        try
        {
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["backupbatch"] == null)
                throw new Exception("backupbatch is not set in App.config");
            sResult = config.AppSettings.Settings["backupbatch"].Value;
            if (!File.Exists(sResult))
                throw new Exception("unable to locate file, " + sResult);
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in GetBackupBatchFile, " + E.Message);
        }
        return sResult;
    }
 
    public static string GetBackupP12File()
    {
        string sResult = string.Empty;
        try
        {
            //determine the code for the current provider
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["backupp12"] == null)
                throw new Exception("backupp12 is not set in App.config");
            sResult = config.AppSettings.Settings["backupp12"].Value;
            if (!File.Exists(sResult))
                throw new Exception("unable to locate file, " + sResult);
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in GetBackupP12File, " + E.Message);
        }
        return sResult;
    }
 
    public static int GetBackupEvery()
    {
        int iResult = 0;
        try
        {
            //determine the code for the current provider
            Configuration config = ConfigurationManager.OpenExeConfiguration(System.Reflection.Assembly.GetExecutingAssembly().Location);
            if (config.AppSettings.Settings["backupevery"] == null)
                throw new Exception("backupevery is not set in App.config");
            if (!int.TryParse(config.AppSettings.Settings["backupevery"].Value, out iResult))
                throw new Exception("unable to convert to int, " + config.AppSettings.Settings["backupevery"].Value);
        }
        catch (Exception E)
        {
            throw new Exception("Exception raised in GetBackupEvery, " + E.Message);
        }
        return iResult;
    }
 
}

Below is the assemblyname.config file

<configuration>
  <appSettings>
    <add key="SmtpHost" value="smtp.gmail.com" />
    <add key="SmtpPort" value="587" />
    <add key="SmtpHostUserName" value="" />
    <add key="SmtpHostPassword" value="" />
    <add key="SmtpUseSSL" value="1"/>
    <add key="FromEmail" value="" />
    <add key="ToEmail" value="" />
    <add key="subject" value="DB Backup"/>
    <add key="backuplocation" value="c:\garbage\"/>
    <add key="backupbatch" value="c:\GPUtilities\cmdbackup.bat"/>
    <add key="backupevery" value="480"/> 
    <add key="backupp12"   value="c:\GPUtilities\ServerBackups.p12"/>
    <add key="backupserviceaccount" value="...@...gserviceaccount.com"/>
    <add key="backupuseraccount" value=""/>
    <add key="backupapplicationname" value=""/>
    <add key="backupprefix" value="BBB_"/>
  </appSettings>

</configuration>

Example of  batch file called in backupbatch for Oracle, this could be anything so long as it ultimately creates a single file %1 passed in from the service.
@ECHO OFF

SET filename=%1.dmp
SET logname=%1.log

EXP credentials FILE=%filename% OWNER=schema LOG=%logname% STATISTICS=NONE

Intel Security, Security or Extortion?

Bing, Yandex, Baidu, Google, Facebook, Sogou, Twitter and Yahoo all do the following; they respect and play by the rules when it comes to crawling sites. When these companies wish to crawl your site, they ask for permission via a request to your robots.txt. These companies respect the sites rules and follow them. Each of these companies identify their crawler with published user agents. These known agents alerts to their identity and intent. These user agent are useful determining if the crawlers seeks the mobile or desktop version of your site. Of course, these user agents are easy to spoof, which is why each of these companies backs their user agent with an rdns entry confirming their identify.

Intel Security / McAfee works by their own set of rules and as a security company pretty much acts like a rogue agent attacking your site. Intel/McAfee does not respect robots. Intel/McAfee uses a generic user agent that is common from bad actor states like the Ukraine, Russia and many countries in Asia; “Java/1.8.0_51” There is no way to distinguish a rogue crawler from Intel/McAfee. After much research we have determined that Intel/McAfee use IP's assigned to such names as pathdefender, or Digital Ocean. How many strikes does a webmaster need to block Intel/McAfee as rogue traffic.

After receiving unwanted traffic from the domain 104.131.0.0 we blocked all of Digital Ocean. Standing up a new site using our common blocked traffic database, Intel/McAfee was blocked as a bad actor in this domain, failing to identifying themselves and using user agents that are known for attacks, they were justifiably blocked.

Here is the rub and where the hubris and true arrogance of this purported security company surfaces. Intel/McAfee is blocked from access to our established sites, yet denied access, they labeled our new site as suspicious, they said they scanned our site and found viruses and malware and they presented a highly inflammatory warning to go back to safety; a load of excrement. The fact is we blocked them, so the message about the malware and viruses they found was a flat out lie.

Through inference and trial and error we determined who was behind the rogue crawls from 104.131.94.252 and 72.5.66.72 with known hacker user agents, spurious DNS; IP's we had blocked as bad actors; Intel! We had no choice but to succumb to Intel/McAfee's extortion; either we allow their rogue agents from unknown IP's onto our site and install their site seal, or they would block our site. Pay up or get blocked, we had no choice but to allow their rogue traffic and install their site seal; extortion.

We find Intell/McAfee to be bad actors that do not properly identify themselves, and if you do not allow their rogue agents onto your site, they will maliciously and without merit label your site as dangerous when in fact they are the problem.

We urge Intell/McAfee to follow the example of Google, Microsoft, Yahoo, Twitter, Baidu, Facebook and Sogou and establish and publish a branded user agent, then back it up with published and identifiable ip addresses via rdns. As security company, shame on you Intel for acting like the very actors you purport to block with your security software.

Seeeduino Stalker v3 Defect

Companies should  pay me to QA their products.  If there is a defect, I will find it.  On my current project, I needed to prototype a pulse counter.  I planned to outsource this, but the Point Six counter itself had so many material defects, namely battery consumption and the counter clock rate too slow,  I decided to build my own.  An EE buddy suggested Arduino.  Arduino is robust, thriving space, I was a kid in a candy store.  "Arduino, I just met a board named Arduino, and suddenly I've found how wonderful a sound can be; Arduino."  Knowing what I wanted, not sure how I would get there, I dove in.  I am not an EE, which my EE buddies love to point out.  EE and CS are kinda of like cats and dogs.  I concede to anyone with an EE, you are smarter then me.

Knowing what I want, lacking confidence to spot a defect, imagine the frustration in discovering the underlying component I was using to prototype has a major design flaw.   Seeed Studio offers a low power board with great components for a battery operated design.  It uses the lower power ATMega328p, has an efficient switching regulator, built-in support for lipo battery and solar charging; delicious.  It also has an XBee socket.  It was the ideal board for my battery operated pulse counter prototype.

Perfect until the wheels feel off.  For whatever the reason, Seeed decided to create an out-of-the box conflict between the serial interface and the xbee; they both used pins 0 and 1.  Out of the box you cannot access your xbee socket and then see what it is doing via the serial monitor.  Clearly Seeed considered this and included "pads" on the back of the board to allow for pin 2 and pin 3 (td/rx) on the xbee socket to remap to P5 and P6 on the mega; great.  Only problem; doesn't work.  Here is where the lack of confidence comes into play.  Seeed introduced a hidden variable into my project, challenging me.

The air was thick around here.  My boxer decided to spend more and more time on the driveway while I yelled and cursed at this board; "why the F isn't this working!!!"   Seeed is in absolute denial, even though this was revealed months ago on their own forum, where they admit to the defect.

Here is the thing; there is an obscure design defect on the Stalker V3.  The P5 pad does not go anywhere, Seeed knows about and they are engaging cavete emptor; let the buyer beware; not cool at all.  The problem is I like this board.  They acknowledged the defect months ago on their forum, yet they have not updated their docs, nor stated how they plan to resolve this problem.  This does help explain when Seeed was blowing these boards out at 80% off a month ago; suckers.

If you are curious, look at the lower left corner of the V3 schematic by the R16 resistor

http://www.seeedstudio.com/wiki/Seeeduino-Stalker_v3

See how the P5 does not attach to the right of the R16 resisitor on the P5 Pad but goes direct to P0 on the mega?   It should look like P6 to the R15 resistor.  The reason this is a such a big problem is the xbee and the uarts both map to p0 / p1 and conflict; you cannot have the uarts -and- and xbee at the same time without remapping the xbee.  This is why they put in the p5 and p6 pads to allow for both uarts and xbee.  Problem is, they didn't catch their P5 mistake and went into to production with the defect.  There is second defect as well, which only adds insult to injury; the through-hole on the xbee does not goto P2 nor P3.  P9 tests ok, but P2 and P3, that path is missing.

This is the only way I could get this board to work properly.

Elegant, don't you think?

SEEED, it is time to recall this board and make this right.  At the very least, disclose this defect.

To 404 or not to 404

SEO is an onion skin, we don't make the rules, we just follow them.  What is a 404 code, this is a response code for a page to notify the crawler that the page no longer exists.  A page with "Page not found" needs to be created server side, in this example IIS, the only place to set a response code is on an aspx pape.  Its not enough to display in the browser, "We are sorry the page you have requested is not found"  You must also in codebehind set the response code to 404, otherwise the crawler sees your page not found as a 200 and indexes page not found.  In the imitable words of Geico, "Everyone knows that."

The question though, are those 404s hurting your page ranking, and the best MBA answer I gave is "it depends."  Part of your page rank is external links to your domain.  Let's say bloggers following your link on your site to hotdomain.com/hothotpage.aspx  Now you decide to change your site and hothotpage.aspx becomes frickenhotpage.aspx.  hothotpage.aspx no longer exists, but all the external links still exist.  Because the external links exists, crawlers keep looking for hothotpage.aspx and you keep returning 404 or if you are really intent 410.  The crawlers keep asking for hothotpage and anyone that follows the external link is getting "page not found."  THIS will effect your pagerank.   External links are essential to your page rank.  Get creative in your IHttpModule and do a 302 to frickenhotpage.aspx.

When to redirect and when to server.transfer.  If the page is truly not found, a typo perhaps, or some bot phishing looking for holes.  hotdomain.com/usercodes.php.   You do not want a 302 to your 404, you just want a 404 to usercodes.php, server.transfer will insert your pagenotfound.aspx with the header for usercodes.php.  But in the case when you are doing a redirect, the case of hothotpage.aspx, you want to notify the crawler of the 302 perm redirect to frickenhotpage.aspx with a normal 200.   This will preserve your external links and page ranking instead of 404 to dead links.  Best practice is to honor the external links and redirect to the new page.

While on this topic, invariably your site is being crawled.  How do you perform system maintenance.  Much like a 302 redirect to a new page with a 200 response.  Its a good idea to redirect all pages during maintenance, or failures with a 302 to a 503 System Unavailable.   This is arguable; server.transfer or response.redirect.  I favor the 302 redirect to a 503 page.

The Firebug plugin is a great way to see how this all works.