Scalable web architectures

A good blog of several leading scaling architectures: Scalable web architectures

Importing data into MySQL

The LOAD DATA statement reads rows from a file into a table at a very high speed. The file name must be given as a literal string.

LOAD DATA [LOW_PRIORITY CONCURRENT] [LOCAL] INFILE 'file_name'
[REPLACE IGNORE]
INTO TABLE tbl_name
[CHARACTER SET charset_name]
[{FIELDS COLUMNS}
[TERMINATED BY 'string']
[
[OPTIONALLY] ENCLOSED BY 'char']
[ESCAPED BY 'char']
]
[
LINES
[STARTING BY 'string']
[TERMINATED BY 'string']
]
[IGNORE number LINES]
[(col_name_or_user_var,...)]
[SET col_name = expr,...]
more...

The mysqlimport client provides a command-line interface to the LOAD DATA INFILE SQL statement. Most options in mysqlimport correspond directly to clauses of LOAD DATA INFILE syntax.
shell> mysqlimport [options] db_name textfile1 [textfile2 ...]
more...

- Palani E

Building SQL Queries - Key points and facts

Below points are DBMS/RDBMS independent and can be applied to any database.
Always try to keep your SQL code simple since tuning a simple code is easier than a complex code and for the database optimizer which chooses the execution plan will function well with less complex code

1. It is always faster to select exact columns than putting * in the select statement
2. Selecting columns which are in clustered or non-clustered index have better performance since the possibility of optimizer using the index is more.
3. Avoid using NOT in the queries since query optimizer ignores indexes and does a table scan.
4. Always try to match comparison condition column sequence with existing index column sequences.
5. Always try to use unique, single –column indexes wherever possible.
6. Avoid using the primary index column in the order by clause when in the select statement primary index column is there.
7. ORDER BY clauses does not use indexes rather they scan the complete table or dataset
8. HAVING clause is used to filter the rows after all the rows are selected. It is just like a filter. Do not use HAVING clause for any other purposes.

9. Sometimes you may have more than one subqueries in your main query. Try to minimize the number of subquery block in your query.
10. Use EXISTS instead of DISTINCT when using joins which involves tables having one-to-many relationship
11. Try to use UNION ALL instead of UNION if the two datasets are unique.
12. To write queries which provide efficient performance follow the general SQL standard rules.
a. Use single case for all SQL verbs
b. Begin all SQL verbs on a new line
c. Separate all words with a single space
d. Right or left aligning verbs within the initial SQL verb
13. Avoid using FUNCTIONS where an SQL index can be used like WHERE clause.
14. Using an equals sign (equi) is the fastest comparison condition if a unique indexes.
15. Always try use LIKE operation without ‘%’ otherwise it will lead to table scanning rather than reading from an index.
16. If the database table has VARCHAR field and should be compared better to go for FULL TEXT INDEXING feature of SQL SERVER.
17. A Join is efficient if when it can use indexes on large tables, or is reading only very small table.
18. INNER JOIN is efficient when index columns are matched together in JOIN clauses. Indexes are only needed if the table is huge. If the table contains small data optimizer might choose table scan rather than reading index.
19. Basic steps to tune JOINS
a. Use equality first.
b. Use range operators where equality does not apply.
c. Avoid use of negatives in the form of != or NOT.
d. Avoid LIKE pattern matching.
20. Use indexes wherever possible except for small tables and join tables on indexed columns.
Check for the Execution plan (SQL Server) if the queries are using proper indexes or doing any table’s scan where they shouldn’t be.

Code Snippets in Visual Studio - Creation & Usage

Visual Studio provides an option of using code snippets in the code behind file. These snippets are readily available to use.

Creating a Code Snippet:
To create a .snippet file
1. On the File menu in Visual Studio, click New and then click File.
2. Click XML File and then click Open.
3. On the File menu, click Save .
4. In the Save as type box, select All Files (*.*).
5. In the File name box, enter a file name with the .snippet file name extension.
6. Click Save.

A line is automatically generated in the file. Below the automatically generated line of XML, add a CodeSnippets element with the proper xmlns attribute value, and a CodeSnippet element to create an individual code snippet.

Following is a code snippet which will provide the set/get property. The Default value is string.

Note: Please remove the '#' before using this code snippet file. Somehow, this blog is not allowing tags [I am thinking of finding ways to make it allow tags and will post it in a separate blog article :)]

<#?xml version="1.0" encoding="utf-8"?>
<#CodeSnippets xmlns="http://schemas.microsoft.com/VisualStudio/2005/CodeSnippet">
<#CodeSnippet Format="1.0.0">
<#Header>
<#Title>Get/Set Property<#/Title>
<#Shortcut$>getset<#/Shortcut$>
<#Description>Code snippet for an automatically implemented property with 'get' and 'set'<#/Description>
<#Author>Vinoth<#/Author>
<#SnippetTypes>
<#SnippetType>Expansion<#/SnippetType>
<#/SnippetTypes>
<#/Header$>
<#Snippet>
<#Declarations>
<#Literal>
<#ID>type<#/ID>
<#ToolTip>Property Type<#/ToolTip>
<#Default>string<#/Default>
<#/Literal>
<#Literal>
<#ID>property<#/ID>
<#ToolTip>Property Name<#/ToolTip>
<#Default>MyProperty<#/Default>
<#/Literal>
<#/Declarations>
<#Code Language="csharp">
<#![CDATA[
private $type$ _$property$;
public $type$ $property$
{
get{ return _$property$;}
set{ _$property$ = value;}
}
$end$
]]>

<#/Code>
<#/Snippet>
<#/CodeSnippet>
<#/CodeSnippets>

This file should be added to "My Code Snippets" folder of Visual Studio. It will be automatically detected and will be included in the intellisense Reflection.

The default path is
"C:\Documents and Settings\{user}\My Documents\Visual Studio 2005\Code Snippets\Visual C#\My Code Snippets"

Using code snippet in the code behind file:
To use the code snippet one has to type the shortcut value specified in the header section of the code snippet.

for the above given code snippet, the shortcut value is "getset".

User has to type this keyword and press the "Tab" key twice.

This will insert the following code

private string _MyProperty;

public string MyProperty
{
get { return _MyProperty; }
set { _MyProperty = value; }
}

Who is a Program Manager

Microsoft is credited to have identified the Program Manager Role in Software Development. A nice write up from one of them about what that role means. The article also has some good links to other articles/books.

http://www.joelonsoftware.com/items/2009/03/09.html

Virtualization

To Build a Virtual Appliance

Why Should You Build a Virtual Appliance?

Save time and money from both a technical perspective and a business perspective when you build a virtual appliance. You’ll lower development and distribution costs, accelerate time to market, expand customer reach, and provide an easy-to-use, more secure software delivery system – all while leveraging industry-leading virtualization platforms. Your customers and channel partners can easily deploy, evaluate, and purchase your solution when it is packaged as a virtual appliance.

Benefit to Software Developers & Hardware Appliance Vendors

Reduce Development and Distribution Costs

Accelerate Time to Market and Expanding Customer Reach

Increase Security and Control

Virtual Appliance Authoring Tools

Below is a select list of virtual appliance authoring tools designed to simplify the creation of virtual appliances, extend the manageability of virtual appliances, and enhance the overall user experience.

VMware Studio

CohesiveFT

JumpBox

rPath

SUSE Studio

**We choose VMware studio as the authoring tool for now

VMware Studio

VMware Studio offers software developers and hardware appliance vendors an easy-to-use virtual appliance authoring tool to create, customize, manage, distribute, and deploy production-ready virtual appliances. VMware Studio leverages the industry's leading virtualization platform, VMware Infrastructure (VI), and offers built appliances all the management services that VI offers

To know more about VMware Studio:

1. Benefits
2. Features
3. System Requirements

**VMware Studio provides a Web Console with appliance templates

Steps to Build a Virtual appliance using Vmware Studio

1. Create a profile for the Virtual Appliance

2. Either your choose OS template or an existing appliance profile

3. Update Appliance Information, Visual customization like logo, label

4. Post Installation tasks like creating/importingdatabases and environment settings could be done with First Boot script. This is script is executed on first boot only.

5. Some maintenance tasks like cleaning up packages, source update automation could be done with Subsequent Boot Script. This script executed on all boots, except the first one.

6. Customize Hardware settings like CPU, memory and hard disk partitions based on the requirement.

7. You could customize the base OS with pre-install & post-install scripts. Specify the password for root user account and add more users if needed

8. Vmware studio has a repository for application packages. We can add custom developed debian packages to the repository which will be included while the appliance is being built. We could add Application Package URL referring to any external resources.

9. Choose application packages fromt the OS media like subversion, MySQL sever & client

10. Specify the Output format and Name for the appliance to be created. ZIP file contains .vmx and .vmdk files which could be deployed using Vmware server. Choose OVF format, to support other range of products like Vmware ESXi server

11. We can opt to enable Update Repository if the appliance needs patches/updates regularly. generated Appliance will access this repository to get updates. ISVs should publish OS and application patches to this repository

12. Need to provide Vmware server settings which will be used while generating the appliance.

13. Once information relevant to the appliance profile are provided, validate the profile to see check the Vmware server, Application packages URL availability

14. After successful validation, select Save and Build Appliance to save the profile and start generating the appliance. Vmware Studio provides verbose log while generating the appliance.

15. Vmware studio provides log history corresponding to various build for that appliance

16. On successful build, you could find links for the generated appliance in output formats chosen(OVF & ZIP). Download the appliance with the link provided and deploy it.


- Suresh Kumar J

Raising event from a user control to a page

User Controls are used very often nowadays. All those controls that would appear on various places as a group are placed as a user control and used as a single control. As the usage becomes easy there are few problems along with this.

The following example explains one such problem and a solution for the same.

An aspx page, say main.aspx, contains three user controls say A,B and C and one button(btnSave) to save the changes. A and B contains various input elements like textboxes and drop down lists. C contains a link which will navigate to some page.

When the user clicks on btnSave the data from A and B are saved to the database. Now what if the user clicks on a link on C, without saving the data? This results in a data loss as the user is redirected to some page without saving the data. This has to be handled in the code to avoid data loss.

The approach would be to save the data on clicking the link button. But how to save data present in other user control from one user control? It is achievable but one has to rewrite the code. To avoid this rewritting, we can make use of events and solve the issue.

The approach is to raise an event from the user control on clicking the link button which will call the btnSave_Click method in the aspx page. This results in saving time for rewritting the code.

following is the code to create an event.

public event EventHandler userControl_event;

this eventHandler will be used to call the btnSave_Click event present in the aspx page.

to associate the btnSave_Click to the user controls eventhandler we have to add the following to the aspx page ( adding to the page_load method will be better)

this.userControlC.userControl_event += new EventHandler(btnSave_Click)

now to raise the event we have to add the following code to the link button's Click event

userControl_event(sender,e);


So, the code of the aspx page and the userControlC will look like following


main.aspx
------------
public partial class main_class
{
protected void Page_Load (object sender, EventArgs e)
{
this.userControlC.userControl_event += new EventHandler(btnSave_Click);
}

protected void btnSave_Click(object sender, EventArgs e)
{
/// User Code to save the data from other user controls
}

}


usercontrolC.ascx
-----------------------

public partial class usercontrolC
{
public event EventHandler userControl_event;

protected void Page_Load (object sender, EventArgs e)
{

}

protected void linkButton1_Click(object sender, EventArgs e)
{
userControl_event(sender,e);
}

}

JS: window.event not working in FireFox but works well with IE

There was a search page which takes three inputs and there are two buttons (one for Search and one for AddNew). When the user types something on textbox and presses "Enter" key, the search functionality has to be called. to achieve we used a Javascript Function which will handle the OnKeyPress event for the textbox.

It is working fine in IE whereas in FireFox it is not working and for the enter key press, the AddNew functionality is invoked.

The following is the js function.

function noenter() {
if (window.event.keyCode == 13)
{
document.getElementById('<%=cmdSearch.ClientID%>').click();
return false;
}
}

this is how it is called for onkeypress event of the textbox.



Issue:

The issue is that the event is not recognized by Firefox and IE was able to detect the event with the help of the window.event object.

Workaround:

The workaround will be to pass the event from the textbox to the javascript method. So the change in the javascript function will be as follows

function noenter(e) {
if (e.keyCode == 13)
{
document.getElementById('<%=cmdSearch.ClientID%>').click();
return false;
}
}


and the onkeypress event of the textbox will be as follows



Thus, Firefox is now able to recognize the event. :)

Programmatically Debugging / detecting Defects using C# Programs

This methodology can be used when the result of a defect is known but it is not known how to routinely reproduce the issue

Its hard to detect defects when we do not know the routine in which it occurs. In simpler terms, if we do not know the sequence and order precedence for occurrence of a defect or bug, then it will be difficult to reach the code to fix the defect.

Though Visual Studio provides excellent debugging environment, at times it will be difficult for developers to detect the defects. For those cases and to reproduce the exact set of events under which the abnormality is occurring. Solving the issues usually requires additional information. This is where the System.Diagnostics Debug and Debugger classes are tools you can use to provide targeted information.

Using System.Diagnostics.Debug

The System.Diagnostics.Debug class provides method calls and properties that can provide formatted conditional information. Code containing the Debug class can remain in release code as it is not compiled into IL unless the DEBUG compilation attribute is set

Useful Methods in this class are

1. Assert
2. Equals
3. Indent
4. Unindent
5. WriteLineIf


example: Debug.Assert(inputValue < Byte.MaxValue, "Value is out of range");

Using System.Diagnostics.Debugger

The System.Diagnostics.Debugger class communicates with an attached debugger. When an assembly is executed within an attached debugger, the Debugger.Break method will stop execution as if a breakpoint is hit by the IDE.

Migrating Emails from Apple Mail to MS Outlook/Outlook Express

Simple steps to migrate emails from Apple Mail to Outlook Express/Outlook

Going from Thunderbird or Outlook to Apple Mail is easy. Recently, we had to go the other way from a Mac to Windows. Could not find a foolproof way to migrate. This is what we have found to work.

1. Convert individual .emlx files to mbox file

• Download the “emlx to mbox Converter” tool. (Click here to download)
• Add a set of .emlx files to be converted either by drag and drop or Open dialog window
• Select the destination folder for the mbox file output
• Start conversion using the “Save mbox “ button

2. Convert mbox file to .eml files

• Download the “IMAPSize” tool which is a freeware Windows application. (Click here to download)
• Install the IMAPSize tool using the installer provided
• Once installed, Go to Tools menu and then run “mbox2eml” module
• Select the mbox file to be converted(Type: All files) using the Open dialog window
• Individual emails will be generated to the specified location

3. Repeat the process till all emails are converted

4. Import .eml files to Outlook express
Drag and drop all generated emails to Outlook Express or Windows Mail(Vista)

5. Import mails from Outlook express to Outlook, if required.