Another great day today, a lot of what I did today built on from what I did yesterday with the 360 degree image, powered by MarziPano, seeing what the graphic communication students had been doing and finally beginning to develop a framework that allows us to create a website where each page is effectively a cell/square on a grid. More on this later!

Developing the 360 degree image prototype

I made some modifications to the 360 degree image prototype today:

  • I used HDRsoft Photomatix Pro to apply some HDR toning to the image and make it look more dramatic. Jamie (my course tutor) suggested that for the final prototype we use a 360 degree image of Burgh Castle (should we decide to implement this experience there) with similar styles applied to it to show that the image in the real web app would not be a realistic depiction of what is there today.
  • I added several more hotspots, notably modifying the HTML and CSS of the ‘reveal’ type of dialog so that each one of these now has a common class that it shares and a unique ID, making it easier to create multiple copies of these hotspots since they now all share the same styles.
  • Personal preference: I don’t much like inline CSS and JavaScript, so I moved all of the CSS and JavaScript into separate files and was careful to keep the code execution in the same order as to not hinder performance.
  • I changed the colour, font and styling of the ‘reveal’ hotspot elements. This was really easy to do – after all, it’s just CSS.

These changes are now available to view on the live prototype, which is available to view here.

Applying HDR toning to the 360 degree in HDRsoft Photomatix Pro.

 

Additions I added included changes to the style of the ‘reveal’ hotspots (they’re now green with an oval picture) and additional hotspots, such as this one about The Ideas Factory.

Testing the 360 degree image prototype across a range of devices

I’ve tested this on the following devices:

  • Samsung Galaxy S8 (2017). Android 9, Google Chrome 72
  • Microsoft Surface Pro 4 (2015), Windows 10 Pro x64 1809, Google Chrome 72
  • Nokia Lumia 925 (2013), Windows Phone 8.1, Mobile Internet Explorer 11
  • Nokia Lumia 930 (2014), Windows 10 Mobile 1607, Edge 38

It worked fine on all of these devices – apart from the hotspots not working too great on the two Windows Phone devices. As I’ve mentioned before though, Windows Phone is a dead platform so it’s not a big problem that it doesn’t work amazingly on these.

Naomi and I spent some time today testing the prototype on the following device and browser combinations. Naomi kindly compiled the results of our testing for me. Our testing method was very simple – open the prototype in the browser, move the device around to test the gyroscope and tap on the hotspots to see if the information showed. We also tried the ‘touch’ mode too for those who didn’t use the gyroscope.

Device Release year and range position Operating system Browsers tested
Apple iPhone 7 2016, high-end iOS 12 Chrome 72

Safari

Firefox 65

Apple iPhone 7 Plus 2016, flagship iOS 11 Safari
Samsung Galaxy S7 2016, flagship Android 8 Chrome 72

Firefox 65

Samsung Internet Browser 7.4

Samsung Galaxy J5 (2017) 2016, low-mid range Android 8 Chrome 64

Samsung Internet Browser 6

Apple iPad (5th Generation) 2017, high-end iOS 11 Chrome 66

Safari

Samsung Galaxy Tab S2 (7″) 2017, mid-range Android 8 Chrome 69

Firefox 69

Samsung Internet Browser 7

The prototype worked perfectly on each of these, the only thing we discovered was that when used with the Samsung Internet Browser with the browser set to the dark mode, the colours in the ‘reveal’ hotspot panels would also become darker. The screenshot below shows the original colour (blue) and the screenshot beneath that shows how this colour is interpreted when the browser is run in dark mode.

How the ‘reveal’ dialog box looks ordinarily.

 

The same dialog box shown in the Samsung Internet Browser with the browser set to ‘dark mode’ (device: Galaxy S8).

We noted that the prototype performed quickly on each device and was an enjoyable experience on each. The dialog windows for the hotspots were big enough and text was easy to read, however we felt that only a small amount of information could be displayed in this experience due to the device screen size being the limiting factor.

The 360 degree image prototype running on a number of devices: Top left-right: Samsung Tab S2 (7″), Surface Pro 4, iPad 5th Gen. Bottom left-right: iPhone 7, Samsung J5 (2017), Samsung S7.

 

Me, positioning the devices in the photo above ready to photograph. Image courtesy Ameer.

The use of the 360 degree prototype

The prototype is hopefully going to be an evolutionary one – meaning that it evolves into what could eventually be considered a final product. The fact that it is coded and not made in something in Axure or Sketch cements this. The fact that it works so well across multiple devices of varying ages and is so responsive is also a massive positive, as is the fact it’s easy to change and adapt. This can definitely be developed further.

The 360 degree prototype proves however that such an experience needs to be a ‘sign-post’ for information, rather than an experience that gives the user reams of information to read. The small size of the hotspots means that only ‘bitesize chunks’ of information can be shown.

  • The small hotspots can only house around 10-20 words of information.
  • The ‘reveal’ hotspots can house around 25-30 words of information – and a picture.

This got me thinking about how more information could be displayed:

  • Audio could be used – perhaps the user could have the option to have more information read to them. Audio would already be a good way of enhancing the app experience by providing authentic sounds as well as authentic imagery from the period that the image is meant to replicate.
  • There could be a larger panel displayed when the user taps on the hotspot, probably at the bottom of the phone (portrait orientation) or the right of the screen (landscape orientation) that could house a lot more information. This could work, but the user may expect the information to appear right above where they tapped to activate the hotspot, not at some other location on the display.
  • ‘Deep linking’ could be used – we could use the website that we are going to build to house information about the different locations and the user could have the option in the 360 image app to ‘read more’, which would redirect them to the website. However, this takes the user away from the experience and they may not return.

These details aside, this prototype is impressive and has certainly ‘wowed’ a lot of people – so it seems exciting and interesting to them! Hopefully the application can be used somewhere.

‘Website on a grid’ framework prototype

The 360 degree image is just one of the experiences that we want to build. Another idea is to have some kind of app, possibly a timeline, where the user reads information on one screen and then can tap buttons or swipe/drag to move the view and see information on other screens. This is really difficult to explain in words, so to make it easier to understand I’ve managed to come up with an analogy: imagine a website where each page is a cell/square on a grid and you can navigate between these cells/squares by pressing buttons or swiping the website.

In the example of a history/timeline app, the rows of the grid could represent the time period (e.g. Victorian era, Tudor era, Medieval era and so on) and the columns could represent specific decades or key years in that time period, for example in the Victorian period column 1840s, 1850s, 1860s, 1870s and so on may be the specific decades. The user could swipe downwards to move to a different point in time and swipe left and right to move between decades.

Ameer already designed a prototype on Principle back in February that shows a timeline app using navigation a little like this.

Sketches of how such a ‘grid-based website’ would function.

The benefits of using a grid

  • No need for space-consuming navigational buttons or fiddly hamburger menus.
  • Very natural and intuitive interaction principles.
  • Each screen can have a different background image and be formatted differently, keeping the viewer surprised and making them want to see what’s on the following pages.
  • Eliminates the ‘wall of text syndrome’ where a user is presented with a massive block of text that they don’t want to, or just won’t, read.

Creating the grid

Initially Jamie, Ameer, Naomi and I looked online to find some kind of JavaScript framework that would allow us to do just this, but we couldn’t really find one. The closest we came really was ‘Owl Carousel’ that allows you to build carousels that moves images or divs along the horizontal axis, but not really supporting downwards movement or creating a ‘grid’ website which is what we wanted. So, I decided to write my own framework using jQuery.

The video below explains the code in full and shows more of it than I’ve written here, but essentially the way it works is that 21 separate divs are arranged into a grid that is 7 columns wide and 3 rows tall (in this example – it can be any size). The divs are set to be 100 viewport height tall and 100 viewport width wide – meaning that they fill the entire browser window. By using jQuery, it is possible to create buttons that will scroll smoothly between these divs when clicked – so there is a button for left, right, up and down and the grid will move in the desired direction.

To see the live demonstration, click here.

The live demo linked to above the video is different to the version in the video. There are some changes:

  • In the video I say that the body width can be set to ‘anything you want as the overflow is hidden’ – in the live version the scrollbars are hidden by using Webkit CSS controls because by hiding the vertical overflow in CSS it also disables scrolling, so this doesn’t work. So now the body width is set to the no. of columns * 100 viewport width, essentially meaning that in this example the viewport width is 700vw, because there are 7 columns so the viewport width needs to be 7 times the width of 100vw – the width of one of the columns.
  • By using ‘min-width’ and ‘min-height’ instead of just ‘width’ and ‘height’, it is possible to make viewport measurements responsive. The viewport width and height on a smaller device, like a phone, is much smaller than a larger device like a laptop or desktop display. This means that when you just use ‘width’ and ‘height’ in CSS, the squares of the grids are displayed much smaller and don’t line up correctly, as shown below.
  • ‘currentcol’ is now added to the variable name instead of ‘currentrow’ on the upwards and downwards buttons to make the code more logical to understand.

This works well and it also means that there’s no need for media queries to make this responsive.

To summarise how the code works, below is an example of one of the functions that is activated when one of the four directional buttons is pressed.

//Down
$('#down').click(function (e) {
    currentrow++;
    $('#row' + currentrow + 'item' + currentcol).scrollVertical(); 
    freezeDownwardsRow(); 
    printInfo();
});

The example above shows the code for the ‘downwards’ button.

There is a variable defined earlier in the program called ‘currentrow’ (there’s also one called ‘currentcol’) which is set to 1 by default, because the first square of the grid (upper left corner) is in row 1, naturally. Each time the ‘down’ button is pressed, the value of ‘currentrow’ is incremented because you are moving down the grid – e.g. if you were on row 1 when you press the ‘down’ button, when you press it you will be moving down the grid to row 2, so the value of ‘currentrow’ has to change to 2.

Then, the Div ID of the ‘square’ that you are moving to is constructed by concatenating some strings together. The IDs I used for each of the 21 squares are shown in the diagram – this will make it easier to understand that line of code.

As explained in the video, the word ‘item’ in the Div IDs could be changed for ‘col’ and that would probably be more logical, but follow the dry-run example below:

Value of currentrow = 1

User clicks on 'Down'

Value of currentrow = 2

ID constructed is: '#row2item1' - apply the 'scrollVertical()' method to this line and navigate to 'row2item1' (which is directly beneath 'row1item1')

Run the 'freezeDownwardsRow' function

Run the 'printInfo' function

The ‘currentcol’ variable is concatenated onto the end of ‘item’ in the ID name because when you are moving up and down the columns the column number will always be static – it will never change. Likewise, when navigating from left to right, the row number is static.

The button for ‘up’ uses very similar code, only this time the value of ‘currentrow’ is reduced by 1 each time so that if you were to press the ‘up’ button whilst on ‘row3item1’  (for example), the constructed ID to move to would be ‘row2item1’.

‘scrollVertical’ and ‘scrollHorizontal’ are methods that I wrote in JavaScript that allow the page to scroll smoothly – up or down, or left or right. Below is the code for ‘scrollVertical’ method.

//Scroll smoothly - UP/DOWN
$.fn.scrollVertical = function () {
    return this.each(function () {
        $('html, body').animate({
            scrollTop: $(this).offset().top
        }, 1000);
    });
};

It’s not too difficult to understand – essentially we are just animating the HTML and body elements and using the jQuery method ‘scrollTop’ and setting the speed of this to 1000ms. The method ‘scrollTop’ can also be set to scroll left or right by changing the line:

scrollTop: $(this).offset().top

To:

scrollTop: $(this).offset().left

This means that I had to make another method that was exactly the same, but was called ‘scrollHorizontal’ that scrolled left and right and append that method to the IDs that need to go left and right.

One problem with incrementing and decrementing values and then navigating to those is dealing with the possibility that a user may keep clicking on a button which will increment and decrement the value so that the code tries to access a Div that doesn’t exist. Imagine this scenario:

  • User reaches ‘row1item3’
  • User clicks on the ‘down’ button 4 additional times and the value of ‘currentrow’ reaches 7 (it goes up by 1 each time it is clicked), meaning that the code is trying to access a div called ‘row7item1’ – which doesn’t exist.
  • The user then clicks on the ‘up’ button to counteract this decision, but nothing happens for 5 clicks because the value of ‘currentrow’ is decrementing each time they press the button, but ‘row6item1’, ‘row5item1’ and ‘row4item1’ also don’t exist, so they have to keep clicking until ‘row3item1’ is ‘created’ and then again so that ‘row2item1’ is created and finally the button scrolls the page upwards. Initially I just thought that the JavaScript was running very slowly, but then I realised that this was the problem. A user would give up.

This is where the ‘freezeDownwardsRow’ function comes in. There are four functions, one for each direction.

var rowcount = 3; //set for the number of rows in the grid


function freezeDownwardsRow() {
    if (currentrow > rowcount) { //set for number of rows in the grid
        currentrow = rowcount;
    }
}

A variable called ‘rowcount’ is declared and this is set to the number of rows and then an if statement in a function checks to see if the value of the ‘currentrow’ variable (defined earlier in the program and changes with button presses) is greater than the value of ‘rowcount’ which specifies the number of rows that are in this grid. If the value of ‘currentrow’ is greater, then it will set the value of ‘currentrow’ to that of ‘rowcount’, meaning that when the user keeps clicking on that button, the ‘currentcol’ value remains at 3, in this example. This means that when the user clicks on the ‘up’ button, there’s no need to press it several times before any action happens, because clicking on ‘up’ is going to do the following:

currentrow--;
$('#row' + currentrow + 'item' + currentcol).scrollVertical();

Meaning that because the value of ‘currentrow’ is limited to 3, when the user clicks on that button the value of ‘currentrow’ goes down to 2 and the code is able to construct the ID ‘row2item1’ which does exist and thus the button works right away.

The other functions for the other directions are similar – it’s just that the ones that go left and right check the ‘currentcol’ value.

The final function to discuss is the ‘printInfo’ function. Jamie said that for development purposes we needed to know which ‘square’ we were looking at, so I wrote this little function that just prints some key information to the console.

//Print information to console
function printInfo() {
    console.log('Row: ', currentrow, '| Column: ', currentcol, '| ID: #row'+currentrow+'item'+currentcol, '| Coordinate: ',currentcol, ',', currentrow);
}

This prints:

  • The current row and the column
  • The ID of the ‘square’ that is currently displayed
  • The coordnate

This is all formatted neatly in the console so that it is obvious which square is being displayed.

You can see in this screenshot that the ID of the div and the row and column number of the div match up with the div that is being shown – in this demo that I coded the row and column number is in each div.

The coordinate of the div is also shown – using the old ‘along the corridor and down the stairs’ method. The coordinates are shown on the diagram below.

The coordinates also help us to visualise where we are on the grid.

The printInfo function also helps to show that other functions, for example look at the console log below.

I got to ‘row1item7’ and then clicked the ‘right’ button an additional 18 items. Without the appropriate freeze function, the number after ‘row’ would’ve just kept increasing and eventually reached ‘row1item18’ – which of course doesn’t exist. However, thanks to the function it didn’t increase and it stayed at 7, then when I clicked on the ‘left’ button it immediately went back to ‘row1item6’.

Content goes into the HTML document, which is laid out like so:

For the sake of keeping the code short, this screenshot only shows two rows worth of content and each row only has 3 columns in it.

There is a div class called ‘row’ which contains separate ‘col’ div classes for the individual squares in the column and then inside those are the div classes (with IDs) that contain the content for each square in the grid.

The framework going forwards

There is still more to do. The problems raised in the video have since been fixed, but the main thing to figure out is how to implement ‘snapable scrolling’, meaning that when you scroll or swipe the div should ‘snap’ to the top of the device. On one scroll or swipe the ‘square’ (or ‘div’) should come into view. The video I recorded below explains this better.

The problem is that basically the methods I wrote for the automated smooth scrolling when the user taps on the directional buttons uses the jQuery ‘scroll’ method. This means that anything else that uses this method will break that automated scrolling. Initially the idea I had was to use code like this:

$(window).scroll(function () {

    //execute code here

});

The trouble is, if you put any code in there relating to scrolling to certain divs then the directional buttons will break – they won’t scroll anymore. Other JavaScript solutions that I have found online also do not work.

At the moment, the best bet is to try the new CSS ‘scroll snap’ functionality, which has only just become available in most browsers – so compatibility might be an issue. Chrome 69 was the first version of Chrome to support it – Chrome 69 came out in September 2018. Essentially the way this works is that each ‘item’ is held in a div class and then each section class div is held in a container. The container has CSS properties similar to:

.container {
scroll-snap-type: x mandatory;
overflow-x: scroll;
display: flex;
}

Then each div class has CSS properties like this:

.section {
scroll-snap-align: start;
}

The HTML would be very simple, something like this:

 

 

 

 

 

 

 

 

 

This would mean that each time you scroll, the next div is shown, set by ‘scroll-snap-align: start;’ which pulls the top of the next div to the top of the viewport display.

This could potentially work extremely well, but:

  • I’d need to rewrite the framework to use CSS flexbox rather than the old system I’m using of arranging divs into a grid.
  • Compatibility may be a concern because this is very new code and has only been supported in Chrome – the world’s most widely-used browser – for 4 or 5 months at the time of writing.
  • It may mean that the directional buttons have to go – I’m not sure how the compatibility would work. Since this experience is intended for mobile devices then maybe that’s not such a big issue.

It’s something that I’m going to try and do once I get the time. If it works, we can use it in our prototype! I am determined to make it work!

So, what’s next? Try to code a new version of this using the new CSS scroll snap feature! Then make a timeline prototype – or something similar!

Reflections on the coding

It’s been a lot of fun! I love coding and learning about new technologies and doing all the problem solving – and I don’t get to enough of it on the course really. But lately, the coding for the 360 degree image and this framework has been really fun and has kept me hooked on this project.

Graphic communication students

The graphics students spent today designing some visual identities in terms of colour and fonts and imagery to define the Angles Way. They’ve decided on ‘The Angles Way, My Way’ to be the campaign name and presented us with several different colour options to choose from. The favourite was a pink/red against a dark blue background – it looks very nice and we will be using this in our work too. They wanted to create something a little different to the usual ‘greens and browns’. I like that they’re thinking outside of the box with this and hope that the Broads Authority understand the idea.

Jamie got the students to research design languages such as Google’s Material Design and Microsoft’s Fluent Design to help them come up with the identities and element styles.

As mentioned before, Zach has been working on an ornate font that is based on 14th Century glyphs and also looks like treadmarks from a boot. Hopefully we can turn this typeface into a font file so it can be used in our app – if not we might just need to use images of it.

Chloe’s been working on creating a collage of images that produce a complete image of Herringfleet Mill which looks really cool and could be a great landing screen or loading screen image. Her and Zach went out and bought a lot of magazines today and did a similar thing, trying to create a collage of landscape images to form one big landscape image.

Finally, Corrina was designing leaflets and posters, as well as playing with Zach’s typeface and seeing how to best stylise the text. She explained that having angled images to fit the ‘Angles Way’ theme (literally) and level text would be better than having the text ‘angled’ too for legibility reasons.

I’m really pleased with how the graphics students have really gotten into the project and all have said that they are enjoying it a lot more than their other project, which is with Aviva. We’re all working really well together and getting on great!

We’re all working in our base room at the same desk.

Bibliography

Flack, R. and Valipour, M. (2018). Well-Controlled Scrolling with CSS Scroll Snap  |  Web  |  Google Developers. [online] Google Developers. Available at: https://developers.google.com/web/updates/2018/07/css-scroll-snap [Accessed 10 Mar. 2019].

LePage, P. (2018). New in Chrome 69  |  Web  |  Google Developers. [online] Google Developers. Available at: https://developers.google.com/web/updates/2018/09/nic69 [Accessed 10 Mar. 2019].