Lucene: Multifield searches

We can run multifield searches in Lucene using either the BooleanQuery API or using the MultiFieldQueryParser for parsing the query text. For e.g. If a index has 2 fields FirstName and LastName and if you need to search for "John" in the FirstName field and "Travis" in the LastName field one can use a Boolean Query as such:

BooleanQuery bq = new BooleanQuery();
Query qf = new TermQuery(new Lucene.Net.Index.Term("FirstName", "John"));
Query ql = new TermQuery(new Lucene.Net.Index.Term("LastName", "Travis"));
bq.Add(qf, BooleanClause.Occur.MUST);
bq.Add(ql, BooleanClause.Occur.MUST);
IndexSearcher srchr = new IndexSearcher(@"C:\indexDir");
srchr.Search(bq);

Now if we need to search a single term across either of the FirstName and LastName fields then we can use the MultiFieldQueryParser as follows:

Query query = MultiFieldQueryParser.parse("commonName",
new String[] { "FirstName", "LastName" },
new SimpleAnalyzer());
srchr.Search(query);

Now if you need to search the term that must exist in both the fields then we use the following:

 Query query = MultiFieldQueryParser.Parse("commonName",
new String[] { "FirstName", "LastName" },
new BooleanClause.Occur[] { BooleanClause.Occur.MUST,BooleanClause.Occur.MUST}
, new SimpleAnalyzer());
srchr.Search(query);

Finally if you don’t want a term to occur in one of the Fields (say FirstName) then use:

 Query query = MultiFieldQueryParser.Parse("commonName",
new String[] { "FirstName", "LastName" },
new BooleanClause.Occur[] { BooleanClause.Occur.MUST_NOT,BooleanClause.Occur.MUST},
new SimpleAnalyzer());
srchr.Search(query);  

so if you need to search a single term across multiple fields then use MultiFieldQueryParser, if you need to search different terms in different fields then use the BooleanQuery as shown first

Dynamically creating types using reflection and setting properties using Reflection.Emit.

I came across a requirement where we needed to create types dynamically based on XML Configuration files, so that in the furture new types are required we dont need to update the application again by creating a new class.
The additional requirement was to populate the property names of the class based on the Cml configuration and its values using the Querystring values from the HttpWebRequest.
I earlier thought about using Dynamic methods from .NET Framework 2.0, but that did not fit my purpose since I didn't need to execute methods from the dynamic type that was created.

So here is the code:
I created three helper methods
1. Generates the AssemblyBuilder and Module Builder objects
2. Generates the TypeBuilder object.. which will be used for generating an instance of the dynamic type using Activator.CreateInstance
3. Create properties using Relection.Emit

   1: public class DynamicType

   2: {

   3:  

   4:     public static AssemblyBuilder asmBuilder = null;

   5:     public static ModuleBuilder modBuilder = null;

   6:     public static void GenerateAssemblyAndModule()

   7:     {

   8:         if (asmBuilder == null)

   9:         {

  10:             AssemblyName assemblyName = new AssemblyName();

  11:             assemblyName.Name = "DWBeacons";

  12:             AppDomain thisDomain = Thread.GetDomain();

  13:             asmBuilder = thisDomain.DefineDynamicAssembly(

  14:                          assemblyName, AssemblyBuilderAccess.Run);

  15:             modBuilder = asmBuilder.DefineDynamicModule(

  16:                          asmBuilder.GetName().Name, false);

  17:         }

  18:     }

  19:  

  20:     public static TypeBuilder CreateType(ModuleBuilder modBuilder, string typeName)

  21:     {

  22:         TypeBuilder typeBuilder = modBuilder.DefineType(typeName,

  23:                     TypeAttributes.Public |

  24:                     TypeAttributes.Class |

  25:                     TypeAttributes.AutoClass |

  26:                     TypeAttributes.AnsiClass |

  27:                     TypeAttributes.BeforeFieldInit |

  28:                     TypeAttributes.AutoLayout,

  29:                     typeof(object));

  30:         return typeBuilder;

  31:     }

  32:  

  33:  

  34:     public static void CreateProperty(TypeBuilder t, string name, Type typ)

  35:     {

  36:         string field = "_" + name.ToLower();

  37:         FieldBuilder fieldBldr = t.DefineField(field, typ, FieldAttributes.Private);

  38:         PropertyBuilder propBldr = t.DefineProperty(name, PropertyAttributes.HasDefault, typ, null);

  39:         MethodAttributes getSetAttr = MethodAttributes.Public | MethodAttributes.SpecialName | MethodAttributes.HideBySig;

  40:  

  41:         MethodBuilder getPropBldr = t.DefineMethod("get_" + name, getSetAttr, typ, Type.EmptyTypes);

  42:  

  43:         ILGenerator getIL = getPropBldr.GetILGenerator();

  44:         getIL.Emit(OpCodes.Ldarg_0);

  45:         getIL.Emit(OpCodes.Ldfld, fieldBldr);

  46:         getIL.Emit(OpCodes.Ret);

  47:  

  48:         MethodBuilder setPropBldr = t.DefineMethod("set_" + name, getSetAttr, null, new Type[] { typ });

  49:  

  50:         ILGenerator setIL = setPropBldr.GetILGenerator();

  51:  

  52:         setIL.Emit(OpCodes.Ldarg_0);

  53:         setIL.Emit(OpCodes.Ldarg_1);

  54:         setIL.Emit(OpCodes.Stfld, fieldBldr);

  55:         setIL.Emit(OpCodes.Ret);

  56:  

  57:         propBldr.SetGetMethod(getPropBldr);

  58:         propBldr.SetSetMethod(setPropBldr);

  59:  

  60:     }

  61:  

  62: }

Then to use these helper methods from the Main program I used the following:

   1: if (DynamicType.asmBuilder == null)

   2:     DynamicType.GenerateAssemblyAndModule();

   3: finalType = DynamicType.modBuilder.GetType("Beacon11");

   4:  

   5: TypeBuilder tb = DynamicType.CreateType(DynamicType.modBuilder, typeName);

   6:  

   7: foreach (XElement e in beaconNode.Descendants())

   8: {

   9:     string pname = e.Attribute("qs").Value;

  10:     string ptype = e.Attribute("type").Value;

  11:     DynamicType.CreateProperty(tb, pname, Type.GetType(ptype));

  12: }

  13: finalType = tb.CreateType();

  14:  

  15: Object obj = Activator.CreateInstance(finalType);

  16: // this sets the properties of the just instantiated class

  17: finalType.InvokeMember("bv", BindingFlags.SetProperty, null, data, new object[] { 1.0 });

  18: finalType.InvokeMember("tp", BindingFlags.SetProperty, null, data, new object[] { 2.0 });

  19: //this sets the properties of the type by using values from the querystring

  20: foreach (XElement e in beaconNode.Descendants())

  21: {

  22:     string pname = e.Attribute("qs").Value;

  23:     object value = context.Request.QueryString[pname];

  24:     finalType.InvokeMember(pname, BindingFlags.SetProperty, null, data, new object[] { value });

  25: }

For more information visit : msdn - PropertyBuilder

Schedule a task using C# code

I needed to schedule a task to run every day at 9:00 p.m. in the night. I had an addition al requirement that the task be scheduled only if the FileSystemwatcher alerts us of new files being available for processing.

Thus the files could be recieved anytime during the day, but despite that the task should be schduled to run exactly at 9:00 p.m. in the night.

So I used the following code to schedule a task(using System.Threading.Timer and TimeSpan classes)

DateTime d = DateTime.Now;

timer = new Timer(new TimerCallback(Update), null, 
TimeSpan.FromMinutes(21 * 60 - (d.Hour * 60 + d.Minute)), 
TimeSpan.FromMilliseconds(-1));

Note I used TimeSpan.FromMillisecnods(-1) to disable periodic signalling

Compare 2 SQL queries for equality

DECLARE @check1 bigint
DECLARE @check2 bigint

Select @check1 = CHECKSUM_AGG(CHECKSUM(*))
FROM
(
    SELECT *
    FROM dbo.Orders (nolock)
)
AS Source

Select @check2 = CHECKSUM_AGG(CHECKSUM(*))
FROM
(
    SELECT *
    FROM dbo.Orders (nolock)
    WHERE IsSuccessful = 1
)
As Comparison

IF @check1 = @check2
    PRINT 'Queries are Equal'
ELSE
    PRINT 'Queries are NOT Equal'

Note:
If order of rows is different it will not effect the result
It does not do execution plan comparisons, simply checks if the rows returned by the two queries are the same or not

More info

Converting SQL2005 DBTimeStamp to Long for Comparison (Convert from Hex to long and decimal to Hex)

If you need to Convert SQL 2005 DB TimeStamp values to Long then follow these steps:
1. Retrieve the DB TimeStamp value in a Byte Array (8 bytes long).
2. Convert the byte array into a Hexadecimal string
3. Convert the hexadecimal string to a long

private long BytesToLong(byte[] bytes)
{
    string ts = null;
    foreach (byte b in bytes)
        ts += b.ToString("X").PadLeft(2,Convert.ToChar("0"));
    return Convert.ToInt64(ts, 16);
}

There is another method to convert the DBTimeStamp to Long, but I dont prefer this method since it can return negative values. The method is:

1. Retrieve the DB TimeStamp value in a Byte Array (8 bytes long).
2. Use BitConverter class to convert the byte array into long

long result = BitConverter.ToInt64(dbTimestamp, 0); 

Inorder to remedy this use the following to generate the same long value as in step 1:

public static long FromDbTimestamp(byte[] dbTimestamp)
{
    long result = 0;
    if (dbTimestamp != null)
    {
        Array.Reverse(dbTimestamp);
        result = BitConverter.ToInt64(dbTimestamp, 0);
    }

    return result;
}

Finally to convert a long into the correct DBTimeStamp use this:

public static byte[] ToDbTimestamp(long timestamp)
{
    byte[] result = BitConverter.GetBytes(timestamp);
    Array.Reverse(result);
    return result;
}

Passing state to threads via anonymous calls

We already know that we can pass state into to Threads in 2 ways:

1. using ParameterizedThreadStart delegate

2.using global variables(reference or value types) accessible to the main thread as well the instantiated thread. This is also the way to share data amonst multiple threads. Infact the most common way to share data between threads is using static variables where application wide scope is desired.

3.using Thread.QueueUserWorkItem ... This differs from Parameterized ThreadStart delegate since it uses threads from a preconfigured ThreadPool instead of manually creating threads

There is another cool method to pass information to threads is via anonymous methods. For e.g.

static void Main
{
    string mainMessage = "Hello from Main !!!";
    Thread t = new Thread(delegate() { Print(mainMessage);});
    t.Start();
}
static void Print(string message)
{
    Console.Writeline(message);
}

Advantage of using this technique is that, it provides strongly typed access to the parameters passed to the thread. The Target Method can accept any number of arguments, and no casting is required (unlike Parameterized Start where the passed state is an “object”) . The flip side, though, is that you must keep outer-variable semantics in mind

Method Hiding... Polymorphism in C#

In the following code:

interface IBand
 {
     int ID {get;set;}
     string Name {get;set;}
     void GetStatus();
 }
 public class Band : IBand
 {
     public int ID { get; set; }
     public string Name { get; set; }

     public Band()
     {
         GetStatus();
     }
     public void GetStatus()
     {
         ID = 555;
         Name = "Ring";
         Console.WriteLine("Base Class Called");
     }
 }

 public class ABand : Band, IBand
 {
     public string Update { get; set; }

     public ABand() : base() {}

     public new void GetStatus()
     {
         ID = 655;
         Name = "TEsst";
         Update = "ddd";
         Console.WriteLine("Derived class Called");
     }
 }
 class Program
 {
     static void Main(string[] args)
     {
         IBand band = new ABand();
         band.GetStatus();
     }

 }

We have 2 classes, Band and ABand which implement the IBand interface explicitly.

Now the call to band.GetStatus in Main will display "Derived class called". This will only happen if the Derived class (ABand) also implements the interface IBand explicitly (not in the literal sense in that it does not implement the methods using the IBand.GetStatus syntax)

If the ABand class does not implement the IBand interface explicitly (but implicitly since it inherits from Band which implements IBand) then the above code will output "Base class called"

However note one thing, that when the CTor for ABand is invoked it will invoke the ctor Band class and which will call GetStatus() method from within Band class hence this will output "Base class called" even though the ABand class has implemented the IBand interface explicilty.

Linq To XML : Check for existence of an element

var bookQuery = 
from book in bookXml.Descendants("Item")

let attributes = book.Element("ItemAttributes")

let price = Decimal.Parse((
  book.Elements("OfferSummary").Any() 
  && book.Element("OfferSummary").Elements("LowestNewPrice").Any()
  ? book.Element("OfferSummary").Element("LowestNewPrice").Element("Amount").Value
  : (attributes.Elements("ListPrice").Any()
      ? attributes.Element("ListPrice").Element("Amount").Value 
      : "0"))) / 100

select new {Price = price};

The trick is to use the Elements(“node”) instead of Element(“node”) and then use the Any() function which will return true if an element of that name exists.

Some other LINQ tips:

Some of the extensions that one can use are Intersect,Union && Except

var list1 = new List<int> { 2, 4, 9, 11, 3, 6 };
var list2 = new List<int> { 3, 8, 4, 30, 9, 16 };
var newlist = list1.Intersect(list2);

This returns : 3,4,9
newlist = list1.Union(list2);
this returns : 2,3,4,6,8,9,11,16,30. finally
newlist = list1.Except(list2);
returns : 2,11,6