Mar 21

How to Implement the Automerge feature that is missing from BitBucket cloud

Recently I was involved in a change over from bitbucket server to bitbucket cloud. It was pretty seemless, but there was one major issue – automerge, which is available on bitbucket server, is not available on bitbucket cloud. For many people, maybe that’s not a big deal, but for us it definitely felt like a step back. We had to do TWO pull requests anytime we did a release, one to master from the release branch and one from master back into the core development branch. In the event of a hotfix… yep, two pull requests there too. In a world of automation being used to make life simpler and to make sure the key steps are done, not having automerge just felt wrong to me and all these other people. So I decided to implement a solution and make it as generic as I could so you can use it for whatever projects you have. I chose to utilize bitbucket pipelines as the driver for this feature. If you haven’t used pipelines you are encouraged to check out their intro here. Let’s check out the solution.

First I describe a more “ideal way”. But I also describe a modified way to implement the same pipeline. The modified way doesn’t require the “bot”… but it’s not as nice. However, it will allow a small group to get things going quick.

Ideal way:

  1. Create a “bot” user – this is really just a dummy user that you will give lots of permissions to so it can do stuff on your behalf.
  2. In the repo, Go to pipelines -> settings
  3. Turn on pipelines.
  4. Go to pipelines -> SSH Keys and generate a set of keys
  5. Copy the public key and go paste it into the bots profile in the ssh keys section.
  6. Go to the repo you want to configure.
  7. Choose the development branch
  8. Give the bot permissions to merge into that branch without a pull request.

Slightly modified way:
If you decide not to go the bot route, you still need to do everything else. You just place the tie the SSH keys you generate into your own user instead. And as long as you have sufficient permissions it will work just fine.

Now that you have pipelines enabled let’s define a simple pipeline that takes whatever is in “master” and merges it to “develop”. The easiest way to it is probably just to copy this one.

# Merges master into development anytime master is updated 
# Is a bit of hack needed because bitbucket cloud does not, at the time of this creation. 
# support automatic merges in their cloud product.
# Built with lots of trial and error. No promises.
# Useful info: https://confluence.atlassian.com/bitbucket/variables-in-pipelines-794502608.html
# More guides at https://confluence.atlassian.com/x/5Q4SMw for more examples.
# Only use spaces to indent your .yml configuration.
# I tried to use variables where possible to keep it generic
# Author: Josh Ahlstrom
# Initial Creation Date: 2019-03-19

# NOTE: Pay no attention to the image utilized below. I chose dotnet, but it probably doesn't matter
#       which one you choose since it is script which means we'll be working "from the shell" anyway.
# -----
image: microsoft/dotnet:sdk

pipelines:
  branches:
    master:
      - step:
         script:
          - apt-get update -y
          - apt-get install git -y
          - mkdir repos
          - cd repos
          # full name variable used to make things generic
          - git clone git@bitbucket.org:$BITBUCKET_REPO_FULL_NAME
          - cd $BITBUCKET_REPO_SLUG
          # variables for git info such as "$GIT_USERNAME" "$GIT_EMAIL"
          - git config user.name "$GITBOT_USERNAME"
          - git config user.email "$GITBOT_EMAIL"
          - git checkout develop
          - git merge master
          - git push

Alright, so now let’s talk about what this does. Keep in mind that a pipeline can be thought of as running in it’s own little linux VM. So you’ve got quite a bit of freedom in what you do.

A quick word of caution: pipelines appear to use “build” minutes. I think they should be called pipeline minutes because you don’t actually have to build in a pipeline, but I digress. If I read it correctly, most accounts come with 50 minutes for free. I ran it a BUNCH of times times getting these scripts set up (maybe 20 times or so) which only totaled about 5 minutes. Anyway, if you have limited minutes that’d you’d prefer to use otherwise then this might not be the way to implement automerge for you – but maybe it’ll give you some other ideas.

Here goes. I’ll show some script and then describe what it does for the whole script.

image: microsoft/dotnet:sdk

First, we tell the pipeline that it’s going to run under an image that knows all about dotnet. There is a docker one, a javascript one… lots of options. I don’t think it matters (as noted in the snippit) which one you use because we’re just doing basic shell commands. It appears that all the images can handle that much.

pipelines:
  branches:
    master:

You can apply different pipelines to different branches, in this case we just want to apply this pipeline to the master branch.

      - step:
         script:

The first (and only) step in our master branch pipeline is to run a script. This just runs it right on the command line BUT has access to certain variables that you can set up in bitbucket, either at the account level or the repo/pipeline level. Some are also global to bitbucket. More on variables later.

          - apt-get update -y
          - apt-get install git -y

The first two lines of the script just install git.

          - mkdir repos
          - cd repos

The next two lines create a directory for us to clone our repo into and move us into that directory.

          # full name variable used to make things generic
          - git clone git@bitbucket.org:$BITBUCKET_REPO_FULL_NAME

The next two lines (including one comment line) actually clone the repo. We use a bitbucket variable here to make things generic and simple to use on multiple repos. The first variable, $BITBUCKET_REPO_FULL_NAME, returns the full name of the repo including the account number.

          - cd $BITBUCKET_REPO_SLUG

Then we “cd” into the repo directory. We use another bitbucket provided variable here, $BITBUCKET_REPO_SLUG. It’s like the shortname of the repo. When you clone a repo from bitbucket and then you look at the directory name it was cloned into… that’s what the value of  $BITBUCKET_REPO_SLUG is.

         # variables for git info such as "$GIT_USERNAME" "$GIT_EMAIL"
          - git config user.name "$GITBOT_USERNAME"
          - git config user.email "$GITBOT_EMAIL"

The next three lines set up the pipeline’s VM git with some made up name and email. This is just because our next step will want to know them.

          - git checkout develop
          - git merge master
          - git push

The last three lines checkout the branch we want to merge into, merge master into it, then then push that branch back up to bitbucket.

We’re almost done!

You might have noticed I used a couple of variables $GITBOT_USERNAME and $GITBOT_EMAIL. Those two I had to create myself. Again, ideally this would just be the username of your real “bot” but even if it is you would want to store it in a variable. You’d just store it in an account variable instead of a repo one. In any case, to set up the variables in the repository you just go to the repository -> settings -> “scroll down” to pipelines -> “then choose” repository variables. Read the little blurb at the top of that page to see how variables work. Basically, don’t include the “$” in your variable name – you only need that to access your variable in scripts.

Finally, to make this work, the “bot” user (or yourself) will need permission to merge into the TARGET branch directly (ie. without a pull request) in order for all of this to work. So make sure you give whichever “bot” / user that got the SSH key you generated earlier permissions to merge directly. You do that in repository -> settings -> branch permissions.

And that’s it! Place this script into a file called bitbucket-pipelines.yml in the top level of your repo and you’re done!

Now when you merge into master, the pipeline should kick off and your automerge should happen as planned!

I hope you found this useful. If you have ideas on how to improve this please feel free to comment!

The seed that spawned this idea came from a similar solution to a similar problem presented here.

Jul 31

Fix Your Error Messages Before You Fix Your Errors

I work in software development. I’m a “Senior Software Project Engineer” which means that I work with other people to define their needs (and wants) and then lead a team that design, architects, and implements a solution. As I’ve moved up over the years I’ve worked with a bunch of different people many of whom are experts in their area. Recently, I got a little bit of verbal lashing from one of these people who is the head of IT Operations. While the person almost never says anything nicely they do almost always end up being right so I try to ignore almost everything they say and instead try to get the message, because, like I said, they are actually almost always right and they’re really good. So what were they right about this time?

Our logging sucks. We log all kinds of stuff. Some of the errors we log are legit. But many, if not most, are total garbage. Some of the “errors” are not really errors (that’s another blog post). But what about those that are truly errors? What could be wrong with logging them?

Imagine an error that says “Fatal error during processing.”

Ok. Now what in the work does that mean?  And what should I do about it? Can I just rerun processing? Do I have to do some kind of clean up first? Should I report it to someone? Was the problem related to the software logic, to the environement (disk space, network down, etc), to the input? What the hell am I, guy who’s job it is to make sure work gets done, supposed to do with that error? I suppose I’ll probably just try to run it again and cross my fingers, right? Well I’m only doing that until someone’s software doesn’t clean up after itself [that’s also another blog post) and causes havoc by being restarted… then I’m out of the business of trying to be helpful and I’m in the business of complaining.

Let’s re-imagine that same error now says “Fatal error during processing – insufficient disk space available”. That’s better, right?! Sure it is. I’d much rather have that! But I still haven’t answered over half of this operator’s questions. Can I just rerun? Do I have to clean up some runtime data first? I t’s better but not really completely helpful.

Trying again. “Fatal error during processing – insufficient disk space available. Process requires atleast 1GB available disk space on volume /server/vol1 to run. Create necessary disk space and rerun.’

Now we’re cooking! All the information anybody needs to have is there in the log. Happy Ops people! And honestly, if you’re DevOps, you probably care about this even more because it’ll be you trying to figure out what went wrong. Logs are important, good logs are a godsend.

So next time you’re fixing an issue and you’re digging through code to try to find the cause of a problem – that you only know about because of a log – keep in mind that most of that digging could be avoided by better log messages. Take the time to update your log messages while you’re in there rather than just fixing the bug. You’ll be glad you did and you’ll make life better for you and for someone else!

Jul 26

VB enums default to 0 if they are Nothing

I was writing a test case today and when doing the first step – “make a failing test” – I was having problem… the test kept passing even though I knew it shouldn’t. Eventually I figured out the reason is was passing is because I didn’t understand that an Enum will default to 0 if is actually Nothing.

I was testing a factory that had the very simple job of producing a logger depending on the type of logger that was requested. Here’s the factory code:

Public Shared Function MakeLogger(ByVal logmode As LoggingMode) As I_CustomErrorLogger
   If logmode = LoggingMode.DATABASE Then
      Return New CustomLoggerDBEnabled()
   ElseIf logmode = LoggingMode.DEVNULL Then
      Return New CustomLoggerDevNULL()
   ElseIf logmode = LoggingMode.STANDARD_IO Then
      Return New CustomLoggerStandardIO()
   Else
      Return New CustomLoggerStandardIO() ' Assume old way if not told
   End If
End Function

And, as you can see, it takes an object “LoggingMode” which is an enum.

My thought was, for my FAILING test I’ll pass Nothing (this is the vb equivalent of null) as the parameter and I can then check to make sure that the type returned is something OTHER than StandardIO (because the default would be StandardIO). Remember I’m trying to make a failing test so I asserted that the returned logger was of type CustomLoggerDBEnabled.

But it passed! That’s not right I said and reran. Then recompiled. Then restarted visual studio literally thinking something must have gotten cached and the test wasn’t rebuilding or something crazy like that. Then it hit me, maybe null behaves weird with enums. So I debugged it and sure enough the value I passed in as Nothing was being view is 0 in the debugger.

If that’s not bad enough, my enums were “auto enums” (I didn’t define a value for them) and so they started at 0, meaning that the first item in my enum was more or less equal to zero, the next equal to one and so on. As it turns out DATABASE was the first item in my LoggingMode enum and therefore it got the value 0. So in my factory, if it was sent Nothing for the enum it would understand that as 0 and therefore translate it to DATABASE when it came to enum comparisons. All of that happening at the same time, and the fact that I just happened to choose DATABASE  as my FAILING condition made the test pass.

The fix was to update my enums to hardcode the value. so instead of:

Public Enum LoggingMode
   DATABASE
   DEVNULL
   STANDARD_IO
End Enum

I did the following:

Public Enum LoggingMode
   ' Today I learned that sending Nothing as a parameter when an Enum is expected defaults the value to '0'
   ' This means that the first item in the list of enums will be selected.
   ' So If we want to send 'Nothing' and have it not match anything in the enums list then we 
   ' have to specify the enum values. This is poop, but it is what it is.
   DATABASE = 1
   DEVNULL = 2
   STANDARD_IO = 3
End Enum

Now, since I don’t have a value mapped to 0 the Nothing enum will fall all the way to my final else statement and give back the default logger I actually want.

To close this up let me point something not quite so obvious out. Besides the in your face lesson of Enums that are nothing are really 0 there’s something else valuable to take away. Unit Testing works to make better code. You shouldn’t skip it. It finds weird little cases more than I’d like to admit. It just so happens that this one gave me an opportunity to learn a little unrealized nugget about the language rather than the implementation. But it has value! Do it!

Feb 06

550.50 On Images in Windows IIS in WordPress

I was having a problem where uploaded images were causing a 550.50 error when users clicked on them to see the original image. That is, images showed up in articles fine, but when the user tried to view the full size version it errored. I was able to fix the problem by setting the permissions of the temp directory that my php.ini file points to such that they matched what wordpress needed. Here’s the two best sources I found on the subject. After the links I’ll post the contents of the archive.org page including all the info you need.

So here’s an attempt at answering the question that did pretty well. It didn’t work for me exactly (I think they got IUSR and IUSRS backwards), but it’s pretty much right. Either way I found this other page that was referenced on that one even more useful.

Here’s that page quoted in it’s entirety because archive.org can have pages dropped and I’m going to want this info later. It’s original location was at the following url. I’m not linking to it because it’s a dead link, but I will link to the primary page for that site.

http://www.digitalpimple.com/tutorials/windows-tutorials/fix-550-rewrite-error

Begin Quote:

While creating this blog I ran into a rather interesting problem that took me all of a half day to figure out. If you installed WordPress on a Windows IIS7 Server using the URL Rewrite Module 1.1, you may receive a HTTP 550 error when clicking on an inserted image.

Well that shouldn’t happen now should it.

After much agonizing I concluded the problem lies in the images’ permissions that seem to get set when first uploading them to/from WordPress. The permissions of the original uploaded image seem to NOT inherit the correct WP upload folder permissions; This in-turn, blocks access to the original file on the server. When the original file is called from the blog/internet, the server throws a 550.50 URL Rewrite Error. The same error that you see in the above image.

Now time for the strange part. Thumbnails of the original are created by WordPress using this same original uploaded image. They display just fine when inserted and called. These thumbnails inherit the correct permissions of the Windows/WP  folder where they are stored. So the question: Why does the original uploaded image that is stored in the exact same location as the thumbnails not inherit the same permissions? Checking the original image directly on the server against the thumbnails that were created from it confirmed my theory. Hum… perplexing. Is this really a URL Rewrite Error or not?

So what causes the original uploaded image from inheriting the correct permissions? The answer is incredibly simple but also incredibly annoying.

The Setup:

For the record, I’m running WordPress 2.8.4 on an IIS7 Windows server 2008 platform using FAST-CGI with PHP, MY-SQL & URL Rewrite module 1.1. I’m using a custom permalinks structure of “/%category%/%postname%/”. The following is the web.config code I’m using along with the same file for download:

<?xml version=”1.0″ encoding=”UTF-8″?>
<configuration>
<system.webServer>
<rewrite>
<rules>
<rule name=”wordpress” patternSyntax=”Wildcard” stopProcessing=”true”>
<match url=”*” />
<conditions logicalGrouping=”MatchAll”>
<add input=”{REQUEST_FILENAME}” matchType=”IsDirectory” negate=”true” />
<add input=”{REQUEST_FILENAME}” matchType=”IsFile” negate=”true” />
</conditions>
<action type=”Rewrite” url=”index.php” appendQueryString=”true” />
</rule>
</rules>
</rewrite>
</system.webServer>
</configuration>

Download: web.config file-1.0

The Solution:

PHP is the issue, not WordPress. The problem only happens when you use PHP to upload a file.  When you upload a file, PHP sends the file to a temporary directory on the hard drive (for me it’s C:\Windows\Temp) and then copies it over to it’s intended directory.  Once the file has landed in the temporary directory, it is assigned the permissions of that directory.  The problem is when Windows copies that file, it keeps the temporary directory’s permissions and doesn’t inherit that of your web directory’s.  Bingo!!

The easiest way to fix this problem is to add to the temporary directory your intended web directory’s permissions.  In other words, don’t erase the permissions already in the temporary directory, just add the web directory’s permissions to them.  In Windows Server 2008 the two user groups you must add are: “IUSR” & “IIS_IUSRS”.

If you want to change your temporary upload directory, find the “upload_tmp_dir” in your php.ini file and set it to the directory of your choosing (outside your web folders of course), and then add the proper permissions.

So, just create a new folder named “PHP_uploads” in  “c:\<YOUR_PHP_DIRECTORY>\PHP_uploads\”. Now go to your PHP.ini file and  change it to the new location.

After adding the new folder location to your PHP.ini file add the IUSR & IIS_IUSRS permissions to the new upload folder you just created.

After your all done, delete the previous uploaded images from your WordPress admin console and reinsert them as normal.

Your 550.50 error is now no more!

Feb 06

Programming Languages Popularity Surprising to Me

I came across the TIOBE Programming Community index a few minutes ago and was a little surprised by what I saw. As developers we all know that there’s much more work being done in different programming languages than what we are exposed to each day. But checking out the TIOBE indicator gives a pretty amazing look at what’s popular and what’s being used in the industry.

The TIOBE indicator is an indicator of the popularity of programming languages. The index is updated once a month. The ratings are based on the number of skilled engineers world-wide, courses and third party vendors. Popular search engines such as Google, Bing, Yahoo!, Wikipedia, Amazon, YouTube and Baidu are used to calculate the ratings. It is important to note that the TIOBE index is not about the best programming language or the language in which most lines of code have been written. – TIOBE.com

 

TIOBE2015jan

Think about it for a moment. Think about which languages you think are the most popular and then click on the link. How did you do? Are the ones that you use on a regular basis the ones that you thought were more popular (they were for me) and were you right (not really for me)? I think it demonstrates a certain level of bias that many programmers have that what we use is probably the best available – unless we’re forced by someone else to use their own favorite choice. It’s important that we be aware of those bias and what’s going on in the rest of the industry so we can grow as developers and remain up to date on our skill sets.

Jan 27

Copy Text from Label – Winforms – DotNet

A customer asked that some of the data on one of their windows forms screens be copy / paste friendly. “Easy enough” you’d think, but on closer inspection I found that the data was being displayed in labels – not text boxes. Nothing wrong with that, but there is no built in way to be able to click on a label and drag your mouse to select the contents of the label so I was tasked with either rewriting the screen to use some sort of disabled textboxes trick or making it possible to click on a label and copy the text of the label to the clipboard.

I chose the copy/paste label route. Here’s what I did. Code follows the steps.

  1. I added a formwide variable that would hold the value to be placed in the clipboard (the value of the label that had been chosen).
  2. Then I added a contextmenustrip to the form and wired up each label to use that contextmenustrip.
  3. When the context menu opens I get the label that launched the context menu and I store the value of that label to that formwide “ToCopy” variable. I also set the text of the context menu option to say both “Copy” and also the value that will be copied to the clipboard.
  4. Finally, I set the ItemClicked method of the ContextMenuStrip to copy the value from the formwide variable (which we set when the context menu opened) to the clipboard.
  5. Consume Beverage of Choice


Dim valueToCopyToClipBoard As String = ""

Private Sub ContextMenuForCopyLabelData_Opening(sender As Object, e As System.ComponentModel.CancelEventArgs) Handles ContextMenuForCopyLabelData.Opening
ContextMenuForCopyLabelData.Items.Clear()
Dim lbl As Label = ContextMenuForCopyLabelData.SourceControl
valueToCopyToClipBoard = lbl.Text
ContextMenuForCopyLabelData.Items.Add(String.Format("Copy - {0}", lbl.Text))
End Sub

Private Sub ContextMenuForCopyLabelData_ItemClicked(sender As Object, e As ToolStripItemClickedEventArgs) Handles ContextMenuForCopyLabelData.ItemClicked
Clipboard.SetDataObject(valueToCopyToClipBoard, True)
End Sub

Modifying the data to be displayed in textboxes would not have been a huge task, but it would be boring and required doing the same exact action a lot of times. Not hard, just a lot of work for a little gain. Plus, textboxes don’t autogrow so longer data would have to be handled some way making it even more tedious. The nice thing about textboxes is that it would have “just worked” without have to handle any special events. My plan if I went the text box route was basically to create a custom textbox control that matched the form, but was readonly, had no border, etc.

Default values would be something like this:

.ReadOnly = true;
.BorderStyle = 0;
.BackColor = this.BackColor;
.TabStop = false;

Then for each textbox that you wanted to hold readonly data and look like a label you’d just use that bad boy.

Each solution is good for a particular situation. If you have data of different lengths going into to your controls then labels are probably the way to go. But in the end it’s up to you.

Hope you find it useful!

The research I did consisted covered essentially these 2 pages:
https://msdn.microsoft.com/en-us/library/system.windows.forms.contextmenu.sourcecontrol%28v=vs.110%29.aspx
(Just a little documentation on how to find out which control actually launched the ContextMenu)

http://www.experts-exchange.com/Programming/Languages/C_Sharp/Q_21693232.html
It turns out my solution looked a lot like theirs in the end, but in my solution we never talk about a specific control so it can be attached to any label (and any number of labels) and just work being written one time.

Have you encountered the same problem and found this useful? Or maybe attacked it a better way. Let me know in the comments below!

Apr 02

How to Copy Files Via scp from Linux to Windows

I recently had a project I took on that involved me wanting to do some super secret stuff. The super secret stuff generated logs, and I wanted to move those logs as “near real time” from my super secret computer to my web server. I didn’t want to give ANYTHING access to my super secret server so it had to be able PUSH data. My jobs were running on a Linux box and the web server was a Windows machine with an ssh server on it (Bitvise SSH – trial edition).

In the end it worked! To get there was a pain, so I’m going to try to hit the high points here to minimize the pain for someone else.

The short version is:

1. Install an SSH server on the windows machine (I used bitvise WinSSHD).

2. Make sure your Linux (I was on linux) has scp, ssh, and all those other goodies.

3. On the linux box run: ssh-keygen -t rsa (do this without entering a password/pass phrase)

What that does is generate a public / private key pair that you can use rather than a password when connecting to a remote machine. So it’ll generate a pair of files – one private key and one public. You want to take the public one and IMPORT it into the the ssh server tool on the target machine under the username you want to connect as. Now when you connect you can skip your password!

So here’s me sending a file from a linux machine to a windows machine. I can still this command in a bash script and that’s all there is to it! It takes the file called cmdFileToCopy in the linux directory /home/ja/  and it uploads it to the windows computer located at “myserver.com”  under the username ‘mywindowsuser’ placing it in the folder “C:\temp”.

# scp /home/ja/cmdFileToCopy mywindowsuser@myserver.com:”C:\temp”

Hopefully you find that useful. Free free to commend below and fill in the details. Over time I’ll come back and flesh this out.

Related reading:

http://www.thegeekstuff.com/2008/11/3-steps-to-perform-ssh-login-without-password-using-ssh-keygen-ssh-copy-id/

http://kb.mediatemple.net/questions/1626/Using+SSH+keys+on+your+server

http://support.suso.com/supki/SSH_Tutorial_for_Linux

http://knowhowshowhow.blogspot.com/2011/07/ssh-tunnel-failures-in-lion.html

Aug 22

Locating Wifi Users Spatially – Are there tools to make it happen?

Okay, guys. The saga continues. In previous installments I’ve discussed my reasons for leaving my wireless access point open (or not open). In general I’d prefer it be open  partially because I want to be a nice guy and partially because I’m lazy and don’t want to have to tell legitimate users (short term guests, family, contractors, visitors) how to connect to the internet. I hate being at the bowling alley and having to ask them how to connect, and even though my house is in a neighborhood if someone needs to connect I don’t really want to have to get involved.

An issue reared its head (again) last night while I was working on one of my websites. All of a sudden my internet speed went to nothing. It was late at night so I thought about just saying “woohoo BED TIME”, but I really wanted to get what I was working on done. Then I got curious. I went and checked my wireless router and saw that there was an extra device making use of it. I’d seen this before and had little slow downs, but it’s usually during off hours and I don’t need full throttle so I don’t worry about it.This time it was somehow affecting my own ability to do work and that is enough to get me agitated.

I didn’t do anything about it because it became an interesting thing to think about… how could I actually find out which person around me it was making use of my connection? Was it my neighbor that is only steps from my house? What about my other neighbor that is a little further away but still within range since there are no obstructions between our houses. Maybe it was an across the street neighbor? Maybe it was someone who wasn’t a neighbor at all and was just parked in the street in the general vicinity. Maybe one of the people that live in the house bought themselves a new cellphone / tablet and didn’t tell me and they’re playing on it or updating an OS or something huge like that. How could I find out?

Well, eventually I fell asleep in my chair and I didn’t come up with an answer, and even today I don’t have one. But what I’m thinking would be cool would be some kind of way of the router being able to tell you the general direction an incoming signal is coming from. My particular router has two antenna so it seems like it’s at least POSSIBLE if the router was smart enough. Or maybe someone out there knows of a cool gadget that I could buy that I could hold in my hand and walk around with that would be a sort of “man in the middle device”. It would be integrated with the router such that it could take over connections and forward them on to the router, but be mobile and be able to actually measure things like traffic speed to the connecting device. So as you carried this gadget around and walked away from a device you’d see the speed drop, and as you walked towards it the speed would go up… sort of a “hotter, colder” game. atleast then I could stand on one of of my house and know if the signal was coming from that direction or not.

I think it would be neat and definitely functional. It would allow me to stroll on over to a particular neighbor’s house and deal with them directly and politely rather than having to just cut it off for everyone or start banning devices.

Have any of you seen such a tool or bit of functionality in an existing router? Or another device that would actually help me solve this mystery? Maybe there’s a device out there that does this an much more. Maybe I’d have to buy three routers that all respond to the same channels and work together as a team and then they could be used to pinpoint all connected devices (spatially). I don’t know, but it’s an interesting thought experiment and something I would have found useful, so I thought I’d put the question out there. If one of you invents such a device now send me a note – maybe I’ll be your first customer!

Aug 01

HOWTO: Using Paypal buttons with ASP.net the easy way

A couple of years ago I wanted to add a donation button to an ASP.net website. Specifically I wanted a Paypal based donation button. It should have been really easy. Paypal will let you make buttons on their site and they’ll generate the code for you to put onto your own web page. But unfortunately… it’s just not really that easy.  I recently needed to do this again (just today actually) and couldn’t bring myself to use one of the old hacks I had done, and I finally found a super easy solution. I don’t understand why it was so hard to find this information, but I decided to write it up for all to see and use.

The problem is that the code that PayPal generate is html form code. The “action” of the form is a page on their website. The reason that this is a problem is because most people use masterpages within their ASP.net webpages and those masterpages typically have form tags already in them. Since webpages are only supposed to have one “form” per page then adding the PayPal generated form in the content area of a page that is wrapped within a form on the masterpage is just not going to work. What happens is that when the user clicks on the PayPal submit button it just posts back to local website instead of posting to PayPal. Very annoying.

I read about 20 web pages / tutorials today about how one can use PayPal generated buttons within an ASP.net, and I was about to finally give up and do it the old way when it occurred to me that maybe I could control the current page’s masterpage’s form’s action programatically. (By the way that was the most possessives in a row ever!)

So here’s the deal, and it is so easy. Let’s say that the PayPal code generated for you looks like:

<form action="https://www.paypal.com/cgi-bin/webscr" method="post">
 <input type="hidden" name="cmd" value="_s-xclick">
 <input type="hidden" name="hosted_button_id" value="blablablabla">
 <table>
 <tr><td><input type="hidden" name="on0" value="Subscription Level">Subscription Level</td></tr><tr><td><select name="os0">
 <option value="Pro">Pro $5.00 USD</option>
 <option value="Allstar">Allstar $15.00 USD</option>
 <option value="Superstar">Superstar $25.00 USD</option>
 </select> </td></tr>
 </table>
 <input type="hidden" name="currency_code" value="USD">
 <input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_paynowCC_LG.gif" border="0" name="submit" alt="PayPal - The safer, easier way to pay online!">
 <img alt="" border="0" src="https://www.paypalobjects.com/en_US/i/scr/pixel.gif" width="1" height="1">
 </form>

Go ahead and paste the generated code into your page wherever it is you want it to go. You don’t have to jump through any hoops here. Now go to your codebehind and add a couple of lines to the “PageLoad” method for the page in question. Here’s a super simple PageLoad with the two new lines:

protected void Page_Load(object sender, EventArgs e)
 {
 if (!IsPostBack) 
 {  } 
 else  {  } 
 Form.Action = "https://www.paypal.com/cgi-bin/webscr";  
Form.Method = "post"; 
}

That’s it! What we did was we just programmatically set our Masterpage’s form’s action and method to whatever the PayPal form’s action / method was. So, now, when you click on the PayPal button your MasterPage form will still be the one that handles it, but it will handle it the exact same way that the PayPal form would have. Enjoy!

Please let me know how it works for you!

Feb 05

Hashing library for iOS and how it can be used in conjunction with .Net hashing

I’m working on an iPhone / iPad app that needs to be able to authenticate itself to the server from which it requests data. The users of the app will have an account with the website so they will already have a username / password. There is no SSL functionality, so I needed a way to authenticate over plaintext without giving away the password or even the hash of it.

Let’s talk about what the webserver does in relation to storing the password. It doesn’t actually store the password.  It stores a hashed value where the actual password was combined with a salt (random string of a descent length) and that combined string was hashed using SHA256. The result is stored as the “password” for the user and the salt used to compute the hash is also stored for that user. When the user logs in the password they send (using ssl, so it is encrypted between the user and server) is combined with the stored salt and hashed. If the result is the same as the stored hash then the user is authenticated, otherwise they are not.

Here’s a little helper class written in C# for doing the Hash. It’s server side and .net specific.


#region HashingLogic
public class Hash
 {
 public Hash() { }
public enum HashType : int
 {
 MD5,
 SHA1,
 SHA256,
 SHA512
 }
public static string GetHash(string text, HashType hashType)
 {
 HashAlgorithm _algorithm;
 switch (hashType)
 {
 case HashType.MD5:
 _algorithm = MD5.Create();
 break;
 case HashType.SHA1:
 _algorithm = SHA1.Create();
 break;
 case HashType.SHA256:
 _algorithm = SHA256.Create();
 break;
 case HashType.SHA512:
 _algorithm = SHA512.Create();
 break;
 default:
 throw new ArgumentException("Invalid hash type", "hashType");
 }
byte[] bytes = Encoding.Unicode.GetBytes(text);
 byte[] hash = _algorithm.ComputeHash(bytes);
string hashString = string.Empty;
 foreach (byte x in hash)
 {
 hashString += String.Format("{0:x2}", x);
 }
 return hashString;
 }
public static bool CheckHash(string original, string hashString, HashType hashType)
 {
 string originalHash = GetHash(original, hashType);
 return (originalHash == hashString);
 }
}
 #endregion

And here’s the webserver checking the Hash. It should be pretty easy to see how to store it as well when a user sets their password in the first place.

 // ran a query that returned the hashed password and salt for the user in question
 // note: password is the value sent by the user trying to log in
 string the_hashed_password = dt.Rows[0]["vcHashedPass"].ToString().Trim();
 string the_salt = dt.Rows[0]["vcPassSalt"].ToString().Trim();
 string the_salted_password = password.Trim() + the_salt;
 string hashedvalue = Hash.GetHash(the_salted_password, Hash.HashType.SHA256);
     if (hashedvalue == the_hashed_password)
         loginSuccessful = true;

 

Now that we have the background, I can talk about what really screwed with me. I was trying to implement part of this on the iOS side. , but I was getting a different value back for my hash. I used a Hash library for iOS from this site. It was easy to use. Just copy the files to your project and import the .h into whatever file you need to do hashing.  Before hashing an NSString you must first convert it to an NSData. But this conversion requires you to specify the encoding. To use it in conjunction with the .Net helper class above you have to choose the right encoding or it will has to the wrong value. Code first, then discussion.

NSString *usernameex = [keychainwrapper objectForKey:(id)kSecAttrAccount];
 NSString *passwordex = [keychainwrapper objectForKey:(id)kSecValueData];
 // I'm hardcoding the salt for this example, now we need to talk to the server  to see if it will let us log in with it
NSString *retrievedSalt = [NSString stringWithFormat:@"b3b5c979-3438-4f56-aa1a-1d85bda78b7e"];
 NSString *saltedpassword =  [NSString stringWithFormat:@"%@%@", passwordex, retrievedSalt];
// Webserver thinks that hashed(password+salt)  =
 // 14083e601f136a42bb17f37a6dedfd03c1f53f2c93f2cc886de5b44ed4a452c1 -- looking for
// but this library was returning various results... here's one
 // ed87d50a50937361f17f9b3c32c6db04e6b7779e5f3c42dfa20e44cac7bf112e -- getting
 // there are lots of example of how the "encoding" can be done and what the hashed value that results is
 // I included them in this sample code because I had to try a ton to finally find the one that matched the .net
HashValue *hashvalue;
//NSUTF16StringEncoding 762ce0d893b64032f11562e4ee0be3f4ed08ca6b801ab892d319af3942ca788c
 NSData* datasaltedpassword0a=[saltedpassword dataUsingEncoding:NSUTF16StringEncoding];
 hashvalue = [HashValue sha256HashWithData:datasaltedpassword0a];
 ASLog(@"hashedvalue0a - %@", hashvalue.description); // password - whatever you entered
NSData* datasaltedpassword0b=[saltedpassword dataUsingEncoding:NSUTF16BigEndianStringEncoding];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword0b];
ASLog(@"hashedvalue0b - %@", hashvalue.description); // password - whatever you entered
NSData* datasaltedpassword0c=[saltedpassword dataUsingEncoding:NSUTF16LittleEndianStringEncoding];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword0c];
ASLog(@"hashedvalue0c - %@", hashvalue.description); // password - whatever you entered
NSData* datasaltedpassword0d=[saltedpassword dataUsingEncoding:NSUTF32StringEncoding];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword0d];
ASLog(@"hashedvalue0d - %@", hashvalue.description); // password - whatever you entered
NSData* datasaltedpassword0e=[saltedpassword dataUsingEncoding:NSUTF32BigEndianStringEncoding];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword0e];
ASLog(@"hashedvalue0e - %@", hashvalue.description); // password - whatever you entered
NSData* datasaltedpassword0f=[saltedpassword dataUsingEncoding:NSUTF32LittleEndianStringEncoding];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword0f];
ASLog(@"hashedvalue0f - %@", hashvalue.description); // password - whatever you entered
//NSUTF8StringEncoding
NSData* datasaltedpassword1=[saltedpassword dataUsingEncoding:NSUTF8StringEncoding];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword1];
ASLog(@"hashedvalue1 - %@", hashvalue.description); // password - whatever you entered
//[NSString defaultCStringEncoding]
NSData* datasaltedpassword2=[saltedpassword dataUsingEncoding:[NSString defaultCStringEncoding]];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword2];
ASLog(@"hashedvalue2 - %@", hashvalue.description); // password - whatever you entered
//NSUnicodeStringEncoding allowLossyConversion:NO
NSData* datasaltedpassword3=[saltedpassword dataUsingEncoding:NSUnicodeStringEncoding allowLossyConversion:NO];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword3];
ASLog(@"hashedvalue3 - %@", hashvalue.description); // password - whatever you entered
//NSWindowsCP1252StringEncoding
NSData* datasaltedpassword4=[saltedpassword dataUsingEncoding:NSWindowsCP1252StringEncoding];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword4];
ASLog(@"hashedvalue4 - %@", hashvalue.description); // password - whatever you entered
//NSMacOSRomanStringEncoding
NSData* datasaltedpassword5=[saltedpassword dataUsingEncoding:NSMacOSRomanStringEncoding];
hashvalue = [HashValue sha256HashWithData:datasaltedpassword5];
ASLog(@"hashedvalue5 - %@", hashvalue.description); // password - whatever you entered
ASLog(@"targetvalue - 14083e601f136a42bb17f37a6dedfd03c1f53f2c93f2cc886de5b44ed4a452c1", hashvalue.description); // password - whatever you entered

And the output:

2012-02-05 16:13:38.077 [11082:560f] LoginViewController.m:121 -[LoginViewController workOfDoLogin] password – ocelot

2012-02-05 16:13:38.078 [11082:560f] LoginViewController.m:140 -[LoginViewController workOfDoLogin] salt – b3b5c979-3438-4f56-aa1a-1d85bda78b7e

2012-02-05 16:13:38.078 [11082:560f] LoginViewController.m:144 -[LoginViewController workOfDoLogin] saltedpassword – ocelotb3b5c979-3438-4f56-aa1a-1d85bda78b7e

2012-02-05 16:13:38.080 [11082:560f] LoginViewController.m:152 -[LoginViewController workOfDoLogin] hashedvalue0a – 762ce0d893b64032f11562e4ee0be3f4ed08ca6b801ab892d319af3942ca788c

2012-02-05 16:13:38.080 [11082:560f] LoginViewController.m:156 -[LoginViewController workOfDoLogin] hashedvalue0b – df7e828a1df0664ff42756960ab7f63e83f0504437c59286e217df32147d8815

2012-02-05 16:13:38.081 [11082:560f] LoginViewController.m:160 -[LoginViewController workOfDoLogin] hashedvalue0c – 14083e601f136a42bb17f37a6dedfd03c1f53f2c93f2cc886de5b44ed4a452c1

2012-02-05 16:13:38.082 [11082:560f] LoginViewController.m:164 -[LoginViewController workOfDoLogin] hashedvalue0d – 4276b8d7ff791492f537379bf7353095d8ce40c717f15dc242f2847b376b88de

2012-02-05 16:13:38.083 [11082:560f] LoginViewController.m:168 -[LoginViewController workOfDoLogin] hashedvalue0e – 45838174fdf6cb95b7ed6bdbd7409f065c75f12958e58ae84ec5d28235b6cf0e

2012-02-05 16:13:38.083 [11082:560f] LoginViewController.m:172 -[LoginViewController workOfDoLogin] hashedvalue0f – 492adfab98090cf9319684aa707d368a2689495480a59ffd7a9b2a514ea9362d

2012-02-05 16:13:38.084 [11082:560f] LoginViewController.m:178 -[LoginViewController workOfDoLogin] hashedvalue1 – ed87d50a50937361f17f9b3c32c6db04e6b7779e5f3c42dfa20e44cac7bf112e

2012-02-05 16:13:38.085 [11082:560f] LoginViewController.m:183 -[LoginViewController workOfDoLogin] hashedvalue2 – ed87d50a50937361f17f9b3c32c6db04e6b7779e5f3c42dfa20e44cac7bf112e

2012-02-05 16:13:38.086 [11082:560f] LoginViewController.m:188 -[LoginViewController workOfDoLogin] hashedvalue3 – 762ce0d893b64032f11562e4ee0be3f4ed08ca6b801ab892d319af3942ca788c

2012-02-05 16:13:38.086 [11082:560f] LoginViewController.m:193 -[LoginViewController workOfDoLogin] hashedvalue4 – ed87d50a50937361f17f9b3c32c6db04e6b7779e5f3c42dfa20e44cac7bf112e

2012-02-05 16:13:38.087 [11082:560f] LoginViewController.m:198 -[LoginViewController workOfDoLogin] hashedvalue5 – ed87d50a50937361f17f9b3c32c6db04e6b7779e5f3c42dfa20e44cac7bf112e

2012-02-05 16:13:38.088 [11082:560f] LoginViewController.m:201 -[LoginViewController workOfDoLogin] targetvalue – 14083e601f136a42bb17f37a6dedfd03c1f53f2c93f2cc886de5b44ed4a452c1

 

The right Encoding to use on the iOS side was “NSUTF16LittleEndianStringEncoding”. I had to do all of those tests to finally find which one matched the values being produced on the web side (.Net)… so just keep in mind that if your getting two different hashes then you might need to use a different Encoding during the conversion from String to Byte / Data in order to make them sync up.