Case Insensitive Lookups with SSIS

Whilst recently compiling a reasonably complex transformation to bring in some data from an Excel spreadsheet I was left in the situation where I had a some look ups that were failing to resolve due to case sensitivity. This was a failure on my part really as I should have understood from the outset that this could have been the case and I should have factored this in from the start but I didn’t. No problem usually, You can just change the case at the source and and in the look-up component and you’re there. However… In my situation this was not feasible without me throwing myself out of the nearest window and making finger paintings with my own blood in my dying moments. The reason for this is that I had about 50 separate sources which is an onerous task in itself. In addition to this as I was ‘unioning’ all of this data together using the union component and then going through several more complex transformations with additional unions along the way it was getting very messy, The union components do not refresh with new fields when the source changes and so the only way to correct this is to drag new union components on in place of the old ones! This in turn means that all of the adjoining components also need to be refreshed and even talking about it I can feel the end approaching.

Read the full blog here.

Written by Conrad Rowlands, Team Leader and Developer, DSCallards  

Visit www.dscallards.com for more information on our Development Services.
 

Happy to do Superficial ...

So…. Sometimes the very simplest of things can take gargantuan efforts to work out and you find out that the detail really is just too much detail.

Just such a  situation happened with me this week when working on a site I am developing. I needed to implement the sending of an email once a post had been received from an MVC page to indicate that a booking had been made. The mail would be sent both to the establishment where the booking was made as well as a copy being sent to the person making the booking. I elected to use the inbuilt System.Net.Mail class, how hard can it be right?…… Well A damn sight harder than I thought that’s for sure! This week there will be no code sample as code was in this instance not at fault or has no bearing on what was happening. To cut a very long and very boring story short it did not matter what I did I could not get the mail class to send the mail; upon sending, the operation would timeout or the server would disconnect etc etc. It made no sense so I took a step back. I had been utilising my Yahoo account using the smtp.mail.yahoo.com server and of course as this was secure and requires authentication I used port 465.

So I thought maybe the issue is with Yahoo and I changed to my Google account instead using the smtp.googlemail.com SMTP server again with port 465. Still no dice…..Odd. Much headbanging later and I stumbled across this StackOverflow article which led me on to this blog.

Now I’m not going to lie and say that it all makes sense to me. It has at least allowed me to at least find a workaround by in this instance using port 587 on my Google SMTP server. If I sat down and thought about it I’m sure I could understand in finite detail what exactly Explicit SSL is and why the .NET mail client only supports this type of SSL, the point is…. I just don’t want to! I feel like I’ve already wasted enough time dealing with this bug (I’m sorry, Its not a feature its a bug whatever they say and one they seem reticent to fix).

Normally I’m a man who likes to get the detail, a superficial overview is never normally enough as it leads to design mistakes. However in this instance I now know in enough detail for me that the .NET mail class does not support all SSL SMTP conversations and I’ll continue to bask in my own ignorance.

Written by Conrad Rowlands, Team Leader and Developer, DSCallards  

Visit www.dscallards.com for more information on our Development Services.

Supporting PHP in Visual Studio

I was very recently asked to lend my support for maintaining a website we have which is written in PHP. Now normally I’d run straight off to my trusty little 2010 macbook (which, by the way, has NOT degraded in speed in all that time nor does it require constant reboots..do you hear that windows? I digress) and use any one of the free or cheap plethora of tools to help with this task. Personally I tend to use Aptana Studio. Just for once I thought I would try and leverage my normal windows work machine for this task and so I turned straight to Visual Studio. It turns out that there is an add-on for studio made by the guys at Devsense. So using the Visual Studio Extensions and Updates manager I installed the add in called ‘PHP Tools for VS 2013′ (which is on a 30 day trial) and awaiting the amazingness…..

Time passed and true amazingness did not happen and so I decided that I would just get on with my work instead.

I downloaded the source for the website put it onto my machine and then started the process. As the project is an already existing site i needed to run the ‘Project From Existing Code’ menu item available from the File/New menu item. This takes you through a simple wizard and at the end of this process you have a visual studio project with all of your PHP code. Great, so now down to it…. Unfortunately what I did not realise was that this site made use of PEAR extensions, thus everytime i tried to run a page with email capabilities it would just break unable to resolve the necessary libraries.

Click here to read full blog.

Written by Conrad Rowlands, Team Leader and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

Augmented Reality

So, IBeacon…. You may or may not have heard of this technology, Indeed up until a year or so ago I must confess that I had not. These funny little items then ended up on my desk, 3 in all, and I wondered what to do with them. As with most things that cross my desk eating them was an option but they looked pretty inedible so after an interval of some months I instead opted to find out what I was supposed to do with them. So i headed over to the Estimote website who are the manufacturers.

 “Estimote Beacons and Stickers are small wireless sensors that you can attach to any location or object. They broadcast tiny radio signals which your smartphone can receive and interpret, unlocking micro-location and contextual awareness.” Instantly I can imagine that sales and marketing teams are practically wetting themselves at yet another opportunity to push their unwanted wares upon us in yet more insidious ways. 

I know that the first and every other subsequent app that tries to promote their wares to me in this way will be removed from my life for ever. I do not need any help in spending money. However looking beyond the more obvious tired use case for these beacons and examining the phrase ‘contextual awareness’ raises some real and positive use cases that I as an individual would actually subscribe to. ‘Contextual awareness’ or ‘Augmented Reality’  is the process of providing supplemental information about an environment or area that would further help the user to understand or interact with his/her surroundings; that is, enriching the real world with appropriate digital information ensuring that that ‘viewer’ has access to a fuller understanding of the environment which they find themselves in. So real world examples…. Well mostly people are trying to sell you stuff.. sigh, you know the drill, you walk into a department store and as you approach the perfume department the stores app will sense your proximity (and odour!?!) and will tell you what special offers are available today.

Click here to read the full blog.

Written by Conrad Rowlands, Team Leader and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

Building Blocks – Objective C

You can only work for so in the world of Objective C before you become exposed to the concept of ‘Blocks’. A very powerful tool in the arsenal of any developer they can drastically increase the amount of code reuse that a developer can leverage in their program. Code reuse is of course a great thing as tried and trusted code can be relied upon to perform correctly and will need the minimum amount of testing whereas duplicated code tends to, as a former colleague of mine used to remark,  lead to the worst kind of inheritance; that is :-  ‘clipboard inheritance’ responsible for many a brainteasing bug. I have also seen it reported that Blocks effectively give you ‘lambda’ in Objective C which as a heavyweight user of LINQ in my preferred language C# I must admit makes me feel a little easier about taking on more Objective C work. They are however a little bit tricky to get your head around when you first come to dealing with them.

Click here to see full blog article.

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards

Visit www.dscallards.com for more information on our Development Services.

Using Visual Studio Templates to Scaffold ‘Framework Stacks’

Any applications developer worth their salt will, even when they have downtime, look for new ways to make themselves more productive whilst retaining their quality levels. One such way that we do this is to have ‘Application Frameworks’, common blocks of code and infrastucture that we use on every project. If we were to write this infrastructure and wiring on every project it would just add to the cost and to the drama.

Click here to see full blog article.

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards

Visit www.dscallards.com for more information on our Development Services.

iPad Orientation Issues

On a recent i-Pad development project using Phonegap and the Ionic framework we ran into issues with the orientation seemingly being set to portrait only. We had been labouring under the somewhat misplaced apprehension that changes in orientation were just supported with no code being necessary.

Upon looking further into this and seeing what other peoples experiences were there seemed to be a lot of people writing handlers to display when the orientation changed such as this:-

Click here for more.

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards

Visit www.dscallards.com for more information on our Development Services.

Where Have You Been All My Life?

Every once in a while a technology comes along that once you have become accustomed to it you wonder how you ever got by without it. For me the ‘stand out by a mile’ example of this is of course LINQ and lambda which if I do not use within any given day means that I’m probably on holiday or procrastinating.

Sometimes, and much more rarely a tech comes to my attention that has been in existence for many years and once used I think…

‘Man, where have you been all my life?’

Read the full blog here.

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

Beautiful South

I have recently been looking into Django and Python as a new direction for my own programming. Whilst the default database migration is quite cool I think that for longer term projects with lots of iterative changes dbSync just does not cut it.
It didn’t take much digging to find mention of a project called South which is a more fully featured database migration tool for grown ups. Having had the migrations work without issue in DBSync I thought that once I had got South installed it would be a simple case of migrating and the job would be, as they say, ‘a good one’. Turns out that South is maybe a little less forgiving than I had thought….

I started off with some of my models defined…. Creating the actual upgrade scripts is as simple as issuing the following command:

$ python manage.py schemamigration MyApplication --initial

A cursory examination of my app and I now see a migrations folder, contained within is a 0001_initial.py file which contains the code required to upgrade my database. Great, without examining too closely I continued to apply the patches to the database

$ python manage.py migrate MyApplication

This too went of without issue and a cursory test of one of the models proved that my changes had indeed been applied:-

>>> from MyApplication.models import MetricIndex
 >>> MetricIndex.objects.all() []

Doesn’t look like much, but the empty block of course means that I have returned an empty array and NOT a big fat juicy error which is good! However my cursory examination of the upgrade script meant that I missed the detail…. I am using abstract inheritance for my tables because with many of the tables they have a lot of common fields. These fields are defined within an abstract model interface and then each table inherits from that. The upshot is I should have 1 database table created for each concrete class and no tables  created for the abstract classes. This was not what the scripts had generated within them. My model classes were defined thus:-

class ModelBase():
 Name =models.CharField(max_length=255)
 Order =models.IntegerField(default=-1)
 class meta:
 abstract =True
class MetricIndex(ModelBase):
 pass

Strange it should have worked… I should have had one MetricIndex table with two fields and no ModelBase table. The answer turned out to be frustratingly simple ( and remember, this worked correctly with dbSync). My abstract class was not defined correctly, ‘class meta’ should in fact have been ‘class Meta’. I applied the changes to the code as below

class ModelBase():
 Name =models.CharField(max_length=255)
 Order =models.IntegerField(default=-1)
 class Meta:
 abstract =True
class MetricIndex(ModelBase):
 pass

I then prepared the database script to migrate to the next iteration of the schema thus…

$ python manage.py schemamigration MyApplication --auto

And applied the schema changes as below

$ python manage.py migrate MyApplication

Voila…. I now had one properly formed MetricIndex table and no ModelBase table. Onward and upward and I urge you to use this great tool!

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards

Sql Server Data Tools Issues

So, there I was beavering away on a Monday morning scoping some changes for some new work for a client. I have recently had a new super quick desktop built for me with Windows 8 (bygones) and all of the usual development tools installed. Visual Studio 2012 is currently our tool of choice twinned with in this instance Sql Server 2012.

In the midst of my scoping I came across some database stored procedures being referenced that I felt unclear about exactly how they achieved their goal. I decided to investigate further by examining the SQL scripts that we keep within our project solution as a matter of course. So I double clicked to open the files as you do and…

Hmm, not what I was expecting to see.



As I previously stated I have recently had a new machine built for me, Visual Studio 2012 and SQL Server 2012 already pre-installed  and I was not expecting such a broken installation. I tried again and did a few more investigations like trying to creating a database project, nope all broken. So I resorted to my most excellent friend, Google, and after a little research here  found that I needed to ensure that my SSDT was up to date. The article spoke about a ‘Check For Updates’ menu item being available from my SQL menu. OK, this was the first clue as I had no SQL menu! I thus downloaded the latest SSDT installation and ran the installation program.

What do you know, it now all works!

I hope this helps.

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

Wireframing with WireframeSketcher

The importance of wireframing an application to allow free and frank discussions with a potential client, cannot I think, be overstated. To this end we use WireframeSketcher. Potential clients generally dont care about the technology behind an application, indeed they dont really care about the controls you use. They just want to know how they will interact with it.

 
Read full blog here.

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

Warm Fuzzy Feeling

Software matters, this much we have ascertained from an earlier blog on ‘that car!’. I think it may be fair to draw a comparison in some ways between something else that really does matter, relationships.  Like any relationship we should be in it for the long haul,  ultimately everybody gets much more out of the software if its not a quick ‘get it done and boot it out of the door’ thing. There’s no feelings of guilt about having done a half job nor the shame of being in it for the one thing, money. In addition there is no awkward history to catch up with you when you least expect it and no sense of the whole relationship being just another logo on your ‘customers’ page .

Read full blog here.

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

Ascertaining Geographic Data within an SSIS Import

So, You probably had it before… A client asks you to plot their addressing data on a map of some sort. Its not an issue you tell them, make sure that within the the data you have the appropriate latitude/longitude data for the address and we will do the rest.. And thats where the awkward silence starts, they haven’t got this data, they NEVER have and data costs!

There are of course ways around this with Google maps of course being top of the list of possible non paying solutions. I thought I’d share my experiences of this with you…. We needed to implement an address look up on an initial import of data using our ETL tool (SSIS). Whilst not being major surgery this is slightly more than ten minutes work …..

Read full blog here.

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

Say What?

So, We have an inherited VB website that we just bought under our own processes and controls. We like to keep things tidy and as a result of this decided to place the website under a Visual Studio solution within it’s own ‘website project’ (not application). There are several other projects associated with the application and as such it makes sense to store them all within one solution, this way anyone who elects to come ‘caped like’ to the rescue does not have to search disparate areas within the whole organisation for the source. All good.

I was then tasked with plugging the solution into our internal build processes (we use CCNet) and after a little scratting around to find how you build a web site the I was hopeful that my work was nearly at end. Upon kicking the automated build off all seemed to go without issue until I got  the following rather spurious error whist trying to ‘Publish’ the website :-

Read full blog here.

Written by:  Conrad Rowlands, Senior Systems Architect and Developer, DSCallards

Visit www.dscallards.com for more information on our Development Services.

Gandalf Gives a ‘Bit’ of Inspiration

So there I was in Camden High Street (which was weird because I live in Devon) posting a letter and minding my own business when a unicorn came along carrying Gandalf the Grey (who was actually my mum in a beard, bygones….) and he spake unto me.

“Young Wizard, have you never thought of using a bit field to represent related data within a database?”

The question threw me, In part because I really like the look of the giant sized marshmallow that Gandalf was now devouring, in part because I wasn’t aware that Gandalf had any interest in RDBS techniques but mainly because I hadn’t ever considered it

Read full blog here.

Written by:  Conrad Rowlands, Senior Systems Architect and Developer, DSCallards

Visit www.dscallards.com for more information on our Development Services.

Float Like a Butterfly ... Errr

I know, I know…., in an ideal world we’d all have the latest and greatest browsers but in the real world companies are not so ‘lithe’ when it comes IT infrastructure.

Recently whilst putting a .NET web project to bed we had some reports from the field of some rendering issues with Internet Explorer 7.  Of course we had tested IE 7 using the compatibility modes present within later versions of IE but seemingly this does not throw up many issues that arise in the ‘real’ browser.


One such issue being reported has been recorded in my blog, here.

Written by Conrad Rowlands, Senior Systems Architect and Developer, DSCallards.

Visit www.dscallards.com for more information on our Development Services.

Theory of Relativity

Recently whilst putting a .NET web project to bed we had yet more reports from the field of some rendering issues with Internet Explorer 7.  Of course we had tested IE 7 using the compatibility modes present within later versions of IE but seemingly this does not throw up many issues that arise in the ‘real’ browser.

I know, I know…., in an ideal world we’d all have the latest and greatest browsers but in the real world companies are not so ‘lithe’ when it comes IT infrastructure.

The issue being reported was that one form displaying an accordion control, which in turn had embedded within it a scrollable div which in turn contained three or four divs as well as a JQGrid was having ‘issues of parentage’… When the main scrollable div was being scrolled all underlying divs were being rendered correctly and scrolling with the content whilst the JQGrid was not moving at all, just merely floating in space and generally getting in the way. After a lot of digging I found the following solution that cured my issues:- The main HTML was defined thus

  with the ‘QuestionnaireGrid’ being the location that the javascript JQGrid is rendered into, The issue however seems to lie with the parent div and not the JQGrid itself, what I needed to do was attach a style via the css to the ‘QuestionnaireGridContainer’ div like thus:

 

 

This ‘position:relative’ style  in this instance seems to explicitly force all children objects to behave correctly with regard to their location within time and space and now when i scroll my div content all child objects are scrolled exactly as I would expect.

Written by:  Conrad Rowlands, Senior Systems Architect and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

Sending Error Mail from Elmah

As part of a recent release of a web application we decided that in order to make the best use of error logs being created by Elmah we should email these to stakeholders as and when they happen to improve the general response times. Here is how we set up ‘that thing’ in our .NET MVC4 application! The red sections indicate properties that were either added or edited to enable the email functionality and green section indicate those fields you will need to change to reflect your choice of smtp server.

Click here for the full blog.

Written by:  Conrad Rowlands, Senior Systems Architect and Developer, DSCallards

Visit www.dscallards.com for more information on our Development Services.

Displaying Alternative Coloured Images in JQ Grid

Whilst coding up a recent .NET MVC application built heavily using JQGrid I came across a requirement which to be honest I struggled with at first . In short I wanted to, on a column by column basis, colorise the icons that were to be displayed. Now JQGrid supports a icon colorset which seemingly cannot be changed easily so in the end I had to resort to changing the styles. Here is how I achieved this, warts and all….


Firstly of course JQGrid uses the standard set of JQuery Ui Icons to provide its user interface. I say ‘Icons’ but this is of course provided as one image which is broken up into sprites of 16 x 16 pixels, more of that later. So the first step is to download the set of colour icons that you require…. JQueryUI.com provides a standard set of these icons rolled in every colour you can dream so its in fact very easy. The hardest part is arriving at your chosen colours!

Click here for the full blog.

Written by:  Conrad Rowlands, Senior Systems Architect and Develope, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

The Piece of Wood and Full Possession of the Facts

 

Now I can’t lie to you, I’m not an insanely practical man….. When it comes to home repairs and the like I’d prefer to pay a man (or woman) who knows what they are doing to do the job; They won’t swear half as much, will generally get the job done without resort to every ill fitting tool in my toolbox, without the use of a calendar and they’ll also have an understanding of the problems that are likely to be encountered on the way. More importantly they’ll plan to avoid the potential issues. (in)Famously I once tweeted that ‘there are only so many ways that 15 pieces of wood can be put together to make a wardrobe, I know because I’ve tried all the others!’

I once knew a guy who was immensely practical, being a general builder for much of his life I asked, and he agreed, to assist me with repairing some windows as I wanted to learn; maybe I could become self sufficient in all my windowing needs and move onto doors and the like… possibly take over the world, I don’t know what I was thinking. Anyway we first needed some strips of wood to replace the pieces that we were going to have to break in repairing the window. How hard could this be? I mean it’s just a piece of wood right, so long as the dimensions are fine and it looks clean…. However I was aware of time passing as we stood in the local ‘shed’ looking at various seemingly identical pieces of strip wood. After he had discarded what seemed like the 7th identical piece I asked what he was doing… I don’t remember his exact response but I do remember that he was particularly keen to ensure that the grain of the wood was absolutely as he needed for a good deal many reasons. It was at this point that my practical life in windows was curtailed as there was clearly way more to think about that I had previously imagined.

I know what you re thinking….. no code snippets in a developer blog? The answer this week is a resounding No! Before one line of code can be written the problem MUST be understood in it’s entirety, you can’t code for a problem until you understand the problem and you can’t code a solution unless you understand how your chosen platform allows you to best solve a problem, well you could but like with the window if we’d used just any piece of wood it would likely have just broken midway through the job.

I guess you can sum this post up in one word ‘Experience.’

Written by:  Conrad Rowlands, Senior Systems Architect and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.

Setting Default Data for Add New operation using .NET MVC JQGrid

Of late I have been working with the .NET MVC JQGrid control for a relatively simple project and have found myself asking all sorts of questions of the technology which do not seem to be easily and readily supported without resorting to hacking around in HTML. One such example came up recently where I needed to default the value for a column upon creating a new Company record. After posting on the forums I was directed to a ‘sample’ which actually showed me nothing even though the forum poster had responded with an answer that was meaningful. I thus decided to document the actual methodology so that you don’t have to!

In short I have a Responder object which upon creation needs the ‘ResponderType’ property to be primed with the value ’2′ to indicate what type of responder we are creating. I have included a full listing  of the ‘MVC’ components although only the items highlighted in bold red are actually pertinent to this blog post. The rest is provided for completeness (although as the code has been derived in reality from some abstracted classes there may be some errors, so apologies in advance).

Click here for the full blog.

Written by:  Conrad Rowlands, DSCallards, Senior Systems Architect & Developer 

Visit www.dscallards.com for more information on our Development Services.

Happy to do Superficial ...

So…. Sometimes the very simplest of things can take gargantuan efforts to work out and you find out that the detail really is just too much detail.
 
Just such a  situation happened with me this week when working on a site I am developing. I needed to implement the sending of an email once a post had been received from an MVC page to indicate that a booking had been made. The mail would be sent both to the establishment where the booking was made as well as a copy being sent to the person making the booking. I elected to use the inbuilt System.Net.Mail class, how hard can it be right?…… Well A damn sight harder than I thought that’s for sure! 

This week there will be no code sample as code was in this instance not at fault or has no bearing on what was happening. To cut a very long and very boring story short it did not matter what I did I could not get the mail class to send the mail; upon sending, the operation would timeout or the server would disconnect etc etc. It made no sense so I took a step back. I had been utilising my Yahoo account using the smtp.mail.yahoo.com server and of course as this was secure and requires authentication I used port 465. 

So I thought maybe the issue is with Yahoo and I changed to my Google account instead using the smtp.googlemail.com SMTP server again with port 465. Still no dice…..Odd. Much headbanging later and I stumbled across this StackOverflow article which led me on to this blog. 

Now I’m not going to lie and say that it all makes sense to me. It has at least allowed me to at least find a workaround by in this instance using port 587 on my Google SMTP server. If I sat down and thought about it I’m sure I could understand in finite detail what exactly Explicit SSL is and why the .NET mail client only supports this type of SSL, the point is…. I just don’t want to! I feel like I’ve already wasted enough time dealing with this bug (I’m sorry, Its not a feature its a bug whatever they say and one they seem reticent to fix). 

Normally I’m a man who likes to get the detail, a superficial overview is never normally enough as it leads to design mistakes. However in this instance I now know in enough detail for me that the .NET mail class does not support all SSL SMTP conversations and I’ll continue to bask in my own ignorance.

Written by Conrad Rowlands, Team Leader and Developer, DSCallards  

Visit www.dscallards.com for more information on our Development Services.

Case Insensitive Lookups with SSIS

Whilst recently compiling a reasonably complex transformation to bring in some data from an Excel spreadsheet I was left in the situation where I had a some look ups that were failing to resolve due to case sensitivity. This was a failure on my part really as I should have understood from the outset that this could have been the case and I should have factored this in from the start but I didn’t. No problem usually, You can just change the case at the source and and in the look-up component and you’re there. 

However… In my situation this was not feasible without me throwing myself out of the nearest window and making finger paintings with my own blood in my dying moments. The reason for this is that I had about 50 separate sources which is an onerous task in itself. In addition to this as I was ‘unioning’ all of this data together using the union component and then going through several more complex transformations with additional unions along the way it was getting very messy. The union components do not refresh with new fields when the source changes and so the only way to correct this is to drag new union components on in place of the old ones! This in turn means that all of the adjoining components also need to be refreshed and even talking about it I can feel the end approaching.

Read the full blog here.

Written by Conrad Rowlands, Team Leader and Developer, DSCallards 

Visit www.dscallards.com for more information on our Development Services.