Donnerstag, 9. April 2015

Binary Search

This exercise is part of my open university homework.

Presented with a brief introduction to the workings of binary search and five code snippets using Pascal, I had to figure out which of these fragments are correct implementations of the given search algorithm.

Binary Search - You're doing it wrong...


Based on the description and after a quick visit to Youtube, I came up with the following idea on how binary search works:


I created a sorted list in ascending order containing 10 random elements: 1, 34, 45, 50, 51, 60, 71, 75, 84 and 99. My search value was 84.

I determined the lists middle element (<list size> div 2) which is 5, and compared its value (50) with my search value. As 84 is greater than 50, I discarded all elements in the lists lower half including 50.

Based on the remaining lists upper half (60, 71, 75, 84 and 99) I kept repeating this halving process until the lists middle element matched the search value.

It took four iterations to find the value 84.


I was fine with my understanding of the algorithm until I ran the first sample implementation using my above list and search value as inputs. Instead of four iterations the program completed the search process taking only three iterations.

This is the given code snippets which...


...I embedded in a little program frame...


...and added some output... 


I was stuck for a while until I realized my faulty assumption: Making the list smaller and smaller each iteration. Implementing this "algorithm", I would either have to remove the discarded elements form the list or copy the remaining elements into a new list.

As my exercise did not mention neither option, this meant back to the drawing board.

Binary Search - You're doing it right...



This image shows the workings of the code snippet:

The field size remains of constant length 10.
The first element is (upperBound + lowerBound) div 2, (10 + 1) div 2 = 5.
As the search value is neither located at position 1 to 5, the implementation increases the value of loverBound by 1, 5 + 1 = 6. Again it calculates (upperBound + lowerBound) div 2, (10 + 6) div 2 = 8.
The value of the eighth element is 75.  As the search element is neither located at position 1 to 8, again the value of lowerBound is incremented by 1, 8 + 1 = 9. (10 + 9) div 2 = 9. The value of the ninth list element is 84, which matches our search value. This makes 3 iterations total.

Having a better understanding of how binary search works, I had a look at the other code snippets.

To determine wether the sample code is a valid implementation of binary search, I have to check if
  • searching is performed according to the given algorithm
  • it terminates properly in case the list contains the search value
  • it terminates properly in case the list does not contain the search value
  • it works properly on a list with length n
  • it works properly on a list with length n + 1

The good, the bad....


As this post already became rather lengthy, I will limit myself to one good and one bad example.

Let's see how the first code snippet (shown above) overall performs:

     Match      No match
length n           x              x
length n + 1                     x              x

This makes it a proper implementation of the binary search algorithm.

The second sample...


...produces the following results embedded in my little program frame:

     Match     No match
length n          x             -
length n + 1                    x             -

In case the list contains the search value, it's found.
But in case the list does not contain the search value, the program does not even terminate.
What's wrong here?

Let's take my list of ten elements (1, 34, 45, 50, 51, 60, 71, 75, 84 and 99) and a search value which is not part of the list, 86.

The first step calculates the lists middle, (0 + 10) div 2 = 5, and compares 86 with the value of the fifth list element, which is 51. So far so good.
Now as the search value is greater than the 51, the value of lowerBound is set to the value of middle, 5. The value of found is false.
The next iteration again calculates the middle (10 + 5) div 2 = 7 and compares 86 with 71. The value of lowerBound becomes 7, and the loop body gets executed another time.
(10 + 7) div 2 = 8. 86 > 75? Yes. lowerBound 8.
(10 + 8) div 2 = 9. 86 > 84? Yes, lowerBound 9.
(10 + 9) div 2 = 9. 86 > 84? Yes, lowerBound 9.
(10 + 9) div 2 = 9. 86 > 84? Yes...................
(10 + 9) div 2 = 9. 86 > ..........................
(10 + 9) div 2 =...............................
................................................
........................................
..............................
.......................
..................
............
........
....
..
.

Sonntag, 8. März 2015

In case of 16-bit, the largest value for an integer is 215-1=32,767....Wait, what?


215-1=32,767...


I signed up for an open university course about Imperative Programming using Pascal. Lecture 1 introduces the integer data type.
"In case of 16-bit, the largest value for an integer is 215-1=32,767 (maxint)."
"The range of possible integer values is [-maxint - 1, maxint]." 
No further explanation given. I picked a couple of programming language books more or less randomly from my bookshelf to check if any of those do a better job at clarifying the upper and lower bounds of a given data type:



The Java Tutorial - A Short Course on the Basics
"The byte data type is an 8-bit signed two's complement integer. It has a minimum value of -128 and a maximum value of 127 (inclusive)." 
Programming Ruby
"Integers within a certain range (normally - 230 to 230 - 1 or -262 to 262 - 1) are held internally in binary form...."
Head First C#
"int is commonly used for whole numbers. It holds numbers up to 2,147,483,647."
You get the idea. Introduce a datatype. Mention the upper an lower bounds. Read, forget, look up again if ever necessary. No deeper understanding gained.

"Objective-C Programming The Big Nerd Ranch Guide" does a slightly better job:
"An unsigned 8-bit number can hold any integer from 0 to 255. Why? 28 = 265 possible numbers. And we choose to start at 0."
"A signed 64-bit number can hold any integer from -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807. One bit for the sign leaves 263 = 9,223,372,036,854,775,808. There is only one zero."

As Big Nerd Ranch's book goes a bit deeper on the subject on how he upper and lower bounds of a datatype are determined based on the number of bits, I assume that I am not the only one out there who doesn't get this right away. I had to explain this to myself...

One more bit...


Let's see. I start simple with a single bit. Its value can be either 0 or 1.
This makes room for two decimal numbers (0, 1).

Binary Decimal
0 0
1 1

I add one more bit.
With two bits it's enough space for four decimal numbers (0...3).

Binary Decimal
00 0
01 1
10 2
11 3

And one more bit.
Three bits can store up to eight decimal numbers (0...7).

Binary Decimal
000 0
001 1
010 2
011 3
100 4
101 5
110 6
111 7

As every additional bit doubles the amount of numbers that can be stored, this is a 2n relation.
2 for the number of possible values a bit can hold (either 0 or 1), n for the amount of available bits.



Upper and lower bounds...


Back to the original Pascal language example. The upper bound for an integer was given as 32,767 (maxint) in the courseware, the lower bound as -32,768 (-maxint - 1). Where do these values come from?

So far I was only looking at positive values including zero. 16 bits would make 216 = 65,536.

What about negative values? The binary system allows the representation of negative (and positive) numbers without requiring additional symbols like "+" or "-". Instead the highest bit is used to indicate wether the number is either a positive or negative one (signed). The value of the highest bit is 0? We are dealing with zero or a positive number. The value of the highest bit is 1? Must be a negative number.

Examples:
  • 0000000000000001: positive number with value 1. 
  • 1000000000000000: negative number with value -1. 

Because in case of negative numbers the value of the highest bit is always set to 1, we have fifteen (16 - 1) remaining slots to fill. 215 = 32,768.

This would cover the range of 0 to -32,767. As by definition zero is part of the positive numbers, we have one additional slot and can go from -1 to -32,768 (-32,767 - 1), which makes -32,768 the lowest number possible (minimum).

Same for positive numbers. The highest bit is occupied by 0. Fifteen remaining slots to fill as well. 215 = 32768. This covers the range of 0 to 32767. Therefore 32767 is the highest number possible (maximum) as we must include the zero.

Positive numbers:

Binary Decimal
0000000000000000 0
0000000000000001 1
... ...
0111111111111110 32,766
0111111111111111 32,767

Negative numbers:

Binary Decimal
1000000000000000 -1
1000000000000001 -2
... ...
1111111111111110 -32,767
1111111111111111 -32,768