Before today, I have written many articles that were concentrated on one topic in SQL Server or Perl or Visual Basic.
Starting today, I am going to kick off a full blown development exercise named PROJECT ALPHA. The purpose of this project is to create a test environment for developing a PERL based Extract Translate and Load (ETL) solution.
The following is a list of business requirements for this ETL project.
- A runnng trace file will be created in the LOGS directory.
- Connect to an FTP server using basic authentication.
- Download a file from the FTP server into a INBOUND directory.
- Remove the file from the FTP server.
- Import the data file into a STAGING table.
- Call a stored procedure to UPSERT the data into the production table.
- Create a directory with today’s date in the ARCHIVE directory.
- Move the data file from the INBOUND directory to today’s directory under ARCHIVE.
All these steps need to be restartable. In short, a job that fails at step 4 can be re-executed without any human intervention.
We will need to create a virtual test environment that has an File Transfer Protocol (FTP) server. The best way to allocate resources such as memory, cpu, and video is to know what hardware you have before giving it away to the virtual machine.
The is a great utility from CUPID called CPU-Z which lists this information.
A screen shot showing 8 virtual cores.
A screen shot showing 4 GB of memory.
Windows explorer can be used to figure out how much space can be used to create a virtual hard disk.
With all this information jotted down in our notepad, we now can move onto the next step, allocating a virtual hard disk from which we will create a virtual machine. Next time, we will be using Virtual Box from Oracle to perform this task.
June 2013 M T W T F S S « May 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30
Cloud TagsAFTER ALTER TABLE ALTER TRIGGER CPAN perl modules CREATE DATABASE create function create procedure CREATE TABLE create trigger create view database administrator database developer DATA TYPES DDL DECLARE DELETE DROP DATABASE DROP TRIGGER DROP VIEW execute EXISTS FORMAT free code INSERT ISNULL John F. Miner III PASS perl script REPLACE SELECT sp_help sp_helptext SQL Server SQL Server Management Studio SSMS. string function sys.databases sys.objects sys.schemas TRIGGERS TRUNCATE TABLE TSQL UPDATE USER DEFINED VIEW vb script