Graye Shattky of Otago Memorybank got in touch with a starting point of requested improvements to Kete. I think they’re an excellent starting point to get a sense of priorities for improving Kete.

The core upgrade is still in progress but it’s a good time to consider what improvements to the Kete product might look like and how they would make your life easier.

I’ve copied their list below. Please drop notes in the comments either commenting on the below features and whether they are the top priorities for you, describing additional features that you think need attention, and whether you think you could support work that would deliver these.

Or feel free to put a document up with your own version of features (you could use old.kete.net.nz or you could share a public Google document).

Graye and his group do great work at Otago Memorybank (http://memorybank.co.nz/). Take a moment and review their archives. He has expressed interest in promoting the project to community funders so feel welcome to comment with any suggestions or offers to help him.

MemoryBank – Desirable Upgrades / Functions in kete

1. Compatibility with mobile devices.  Enable easy search and edit functions for i-pads and androids.

2. Simplify the procedure for adding images within the description text box. Having pressed the camera icon, amend the ‘select image’ pop up window by enlarging/highlighting the ‘Upload new image’ hyperlink and re-locating the ‘search’ button lower on the screen. Having clicked ‘Upload new image’, remove unnecessary editing icons from the new screen and request only ‘title, caption and alignment; there is no necessity to repeat description details which have already been recorded.

3. Enable up-loading of multiple images for use by all contributing members (currently only enabled for administrators using ‘import set’ function).  Ideally, should be able to highlight multiple images and upload as a ‘one click’ function.

4. Video viewing – needs to be quick, simple and compatible with all commonly used systems. Present embedding of Youtube clips requires skills probably greater than those possessed by the average contributor.

5. Slide show viewing – enable the following functions:

• click on the image’ function which increases the image to full screen size (including title and caption)
  
• click in the right  margin’  scrolls forward through the gallery of images;
• ‘click in the left margin’ scrolls back through the gallery.
• Add a re-order function which enables the sequence of images to be changed.

6. Easily ingest images from Social Media (predominantly Facebook)

Memory Bank are finding many community members being happy to scan and share local heritage items on their Facebook wall. They would like a solution to easily accept all images into their Kete collection for archiving.

One of the Memorybank images of Central Otago

Josh’s comments – if you’re having trouble commenting I’ve reported a bug to JD at Horowhenua, the website master.

Compatibility with mobile devices Probably easiest delivered by developing mobile stylesheets – m.kete.net.nz layouts with some mobile optimised functionality. Search functions and add item functions would seem to make sense [requires upgrade, medium cost]
Simplify the procedure for adding images within the description text box. this text viewer is being upgraded anyway. It’s an old text editor. I’m not certain if the default interaction will be as you describe but should be a lot easier [requires upgrade but is included in current roadmap]
Enable uploading of multiple images for use by all contributing members – requires upgrade but low cost – really good idea Graye, I’ll do my best to bundle that in as we’re currently upgrading the user management system
Video viewing – this one is a little more involved. Video online is still a mess and some would say getting worse, not better with modern device fragmentation. Will need to take a look at what can be done in-app. A bit of a hack might be to store the video file and lodge it in Youtube and auto embed the file. This would ride on the coattails of Youtube’s infrastructure – but this would violate the privacy expectations of some Ketes – large but some moderate interim steps or possible hacks]
Slide show viewing
if you’re comfortable having a basic version of this, I’d tentatively say it was low/medium
Easily ingest images from Social Media (predominantly Facebook)
ingestion of the images is fairly easy but how the feature would operate might be tricky. I can see an admin ability to subscribe to a public feed and ingest all new images that appear there. Would rely on Facebook groups being public information. A larger piece of work to make it more robust but I think the light approach could work

Taking a look at a modern library service in the age of Facebook

Kete is getting an upgrade. Founded in 2007 as an open-source project, well over 40 installations of Kete run throughout New Zealand. Since those early days, discussion of the relevance of digital heritage and digital preservation has become more prominent. Many library sector discussions consider involving the community in digital heritage – Kete’s strength is in community-contributed collections.

We recently announced an update to the technical upgrade project and are seeking expressions of interest via the Horowhenua Library Trust for NZ libraries to maintain their Kete instances, and prepare to increase activity in their collections.

Using Kete to realise community value

How might you go about planning a digital heritage campaign in your community and who would be involved? While the open nature of Kete attracts the general public, history indicates local community groups value the project most as it enables cooperation, collaboration and documentation of activities that can be informal and difficult to otherwise record. This might be a local heritage interest group such as the Otago Memory Bank ; it might be an ethnic group of tangata whenua or migrant communities; or it might allow the documenting of Cuba Street history, leveraging multilingual capabilities to consider all perspectives. With a clear project purpose, it’s very compelling to share images, records, and notes, building stories together. In the context of creating diverse digital heritage, these collections can easily be set to be indexed within Digital NZ and can be found by anyone searching for these stories.

Kete also has easy API access to display items around the web, such as within a content management system, social media or in a museum display. This can make local displays and presentations far less technically demanding, when Kete serves as an enduring repository of collection records and assets.

A great example of Kete being used to meet community needs came in the advent of the Rena disaster. Smita Biswas in Tauranga coordinated interested community members to take records of the emerging history of the event. In the context of social media telling the story in real-time, Kete enabled an element of scrapbooking, as a chronicle of perspectives as they occurred. The case study was presented at the LIANZA conference and you can read more here.

Technology Considerations

The project very easily extends to be searchable with Digital NZ. It also can be customised to reference other database results. This flexible sharing of data works well to allow a holistic merging of various databases for use online.

The current generation of the codebase is now outdated and very challenging to manage. The major upgrade underway is a critical step to ensure the software foundations of Kete are improved and stable.

There has not been any dedicated maintenance budget of the software for some time, making this phase the most technically challenging. Rabid have a roadmap of user-focused improvements and features and it makes sense to improve the experience to be easy-to-use and accessible to meet users’ expectations of a web service.

Modernising the software and releasing this as opensource will give complete control over the solution to libraries, but we’re working hard on support packages. There are other considerations to keep the software modern, and we’ve developed a plan to build sustainability into New Zealand’s efforts.

Kete projects do require resource to support and encourage uptake in NZ communities. We would be happy to provide guidance and examples of successful projects to assist your planning.

So, please consider the potential of your library or district using Kete.

Time to Budget for a Kete in your community for the new financial year

Joann Ransom, Horowhenua Library Trust
Josh Forde, Rabid Technologies

Background

Rabid and Horowhenua Library Trust kicked off the ‘Kete2’ upgrade project in August last year.  We put out a proposal to raise $200k and manage a very uncertain upgrade in a way that would allow Kete collections to be upgraded and modernised. All parties aim to upgrade the project to be modern, usable and in a state that new libraries and community groups can run digital heritage projects with confidence.

Stage one is an upgrade/rebuild phase.  Future phases will need to address reaching a sustainable arrangement for New Zealand users and libraries, and subsequently efforts to expand the reach and impact of the project.

Scope

The project scope of the first phase was to target the largest risks to the technology and mitigate security concerns. Over time, a lack of maintenance funding has meant that certain open-source components have lost use and the project hasn’t maintained small upgrades.  Coupled with a codebase that has had an iterative approach, the architecture underlying the software was often sprawling and a real challenge to comprehend. Many Kete instances are doing some creative things with the features of the system and it has been challenging dealing with the breadth of that. The short version is that scope has been much larger than anticipated but we’ve been making steady progress. To simplify the requirement, the project is focusing on the simple collection and description features that a typical library would require.

Progress Update

Rabid are committed to delivering an upgrade.  What this large increase in scope means is:

• planned development resource needs a lot more time as well as the financial resource that is involved. We have a core team of 2 developers, Rob Ramsay and Eoin Kelly, doing most of the work in the upgrade. We’ve utilised pair programming to get better progress refactoring and ensuring we don’t get stuck on the work.

• where Rabid are putting in this time, they are doing this with a lot of commercial risk.  Rabid are essentially putting in the investment to be able to offer Kete users the ability to upgrade, in the hope that libraries will come on board.

• to date (end of Feb), Rabid delivered a lot of work.  We did a calculation of work to progress:

• the commercial development bill is approximately $80,000 of work against $35,000 to date.

• this number doesn’t take account of the analysis, community meetings, relationship building and sales time Rabid put into the project. They don’t seek to recoup this time but it is a cost to them.

• our current estimates of work remaining are that there may be 6-8 weeks FTE workload to come, potentially more.

• we need to balance this workload against commercial realities.  The Rabid team will continue to work on this, but need to ration that workload going forward.

• The core ‘risk’ of this phase has been worked through now.  There is still a lot of work to be done, but the unforeseeable risks of the upgrade itself are done albeit the cost has been at the pessimistic end of our estimates.  We have got enough confidence to have built user stories level analysis and have agreed milestones ordering what we will do from here to deliver.

• There is however some uncertainty about how long it will take to deliver a functioning demonstration site and upgrade the Horowhenua Kete to provide completion.

WHERE IS THE TECHNOLOGY RIGHT NOW?

Our desire is to simplify and standardise the codebase.  We won’t have a project we’ll be entirely satisfied with technically, but we’ll have confidence that the core application has integrity.  There are future decisions to be made around delivering and supporting Kete in a manageable way, but the essential code will be of reasonable quality going forward.

A brief overview of the work completed :

1. Ruby on Rails 3 upgrade has been completed.

The upgrade of this version necessitates upgrading the dependencies, gems, libraries that functioned for previous Kete. This is an amorphous body of work incorporating understanding the architecture, upgrading the systems and tracing errors through the code.

2. The search engine Zebra has been pulled out of the system. This search engine didn’t get uptake in the Ruby on Rails community but PostGres search features have been perfectly adequate for Kete to date.

3. The database has been changed from MySQL to PostGreSQL.  We communicated an intention to use MongoDB, which is the database of National Libraries.  When we upgraded, we found that PostGreSQL met our needs without the added complexity of Mongo.  PostGreSQL has recently become a strong de facto database, being resilient, flexible and widely supported.  This doesn’t mean versions of Kete wouldn’t work with MongoDB but there are more pros for PostGreSQL than reasons to use a NoSQL solution.

4. We have migrated data from Kete Horowhenua in our development.  This will show that the first Kete has been upgraded and allow testing and confidence in the deliverable.  There has been prioritisation of some features, and we have gone for a functioning system over exhaustively rebuilding features.  For example, some past migrations from other databases (such as past perfect) haven’t been tested, as is the template creation of extended fields.

Why is Rabid committing resource to this upgrade?   

Joann Ransom and Josh Forde have been speaking for some time about the project.

There is clearly increasing awareness and conversation about the importance of community in digital heritage and collections.  Rabid believe if we can complete this project, Kete will be able to grow again, including beyond New Zealand.

Kete delivers an excellent service and the collections you have are very valuable already. We don’t blame any particular user for the state of the project – but Rabid see small community projects like Pukekura Park or the Otago Memory project doing great work collecting digital heritage items, without the technical services to keep those projects secure for the long-term.

When balanced with the overhead of providing a well controlled, standardised software solution, Rabid think Kete is of great value and can grow towards a vision of any New Zealander’s digital history being able to be stored, described, and ‘storified’ in a digital format that will outlast the physical media that their history/whakapapa hold.

In short, Rabid believe in the underlying values of Kete and are working to a point that we can give you confidence in the software that runs your collections.

What next

Naturally we would like to deal to these problems immediately, but we need to work within our resources and keep steady progress on the technology. Rabid are taking on as much as they are able, and it would be extremely helpful from here for libraries to make commitments to the project (expressions of interest via Te Horowhenua Trust) for either:

a.    a new Kete subsequent to the upgrade including an allowance for software maintenance and upgrades in a collective fashion
b.    willingness to commit to an upgrade of their Kete, in tandem with a support arrangement.

A pricing model will be available in the next week or so.

That said, it’s exciting to have progress, and to be able to browse and explore the new Kete within a new application framework.

You can view a demonstration site using Kete Horowhenua at kete2.herokuapp.com.  Functionality is strictly browsing only (all creation and editing of items is upcoming)

I asked for a few minutes to address the NDF board at the AGM because it was important that the GLAM community are informed of the uncertain status of the Kete project.

Kete collects digital content, informal, community content, that sits alongside the formal content produced by memory institutions throughout the country.

Kete is 2007 technology. We have not managed to build a sustainable financial model to fund enhancements over the last 6 years.

Do we value citizen created content?

We reached a crisis point about 6 months ago and had to upgrade the rails foundation of Kete because the version Kete was running on it is no longer supported; a massive security vulnerability. Horowhenua has 22,000 digital assets at risk and we asked the community to join us in upgrading Kete. There are 250,000 digital assets stored in various Kete throughout the nation.

A ‘benevolent dictatorship’ was formed to make the urgent decisions about the security upgrade. The team comprises:
Andy Neale – Digital NZ, Leith Harhoff – Palmerston North, Jo Ransom – Horowhenua and Josh Forde – Rabid Technology. Digital NZ contributed $20k, Horowhenua $30k and Palmerston North and Tauranga Libraries each contributed $5k to upgrade the Rails and underlying dependencies to address the critical security issues. There will be a noticeably speed increase.

We now how to decide whether we value citizen created content or not?

Do we abandon Kete or do we upgrade?

We could abandon Kete:

• What will we do with the 250,000 existing digital assets stored in various Kete?
• Do we want to collect citizen created digital content anymore? eg the shoeboxes of photographs stashed under beds,
• If so, what is the tool that will replace Kete? I don’t know of an alternative.

Or we can upgrade Kete:

• We have a road map of development that will create a completely modernised Kete with a strong backend that will support awesome ‘front end’ stuff,
• It will cost around $200k to do the upgrade,
• We need to develop a subscription model to ensure the Kete code doesn’t stagnate again.
  eg If each of the 34 APNK libraries who were funded free Kete through Digital Strategy funding had each paid $2k pa over the last 6 years, and if the city and other libraries who are running Kete had each contributed $2k – $5k a year, Kete would have remained the award winning, world class tool that it was when first developed in 2007.

Horowhenua can no longer continue funding Kete for the nation.

My plea to the board is for advocacy help. There is an urgency about this decision, these digital assets are at real risk and we need the conversations to be held in high places.

A round of notable Kete people were in Wellington for the 2013 National Digital Forum.  It was great to connect.

• Irma of Calyx in Australia, Leith Haarhoff of Palmerston City Libraries, Harley Couper of Tauranga Library were all in attendance
• Penny Carnaby’s keynote referenced Kete as a point of success in NZ’s digital heritage
• The Kete dinner gave us a moment to connect on the project
• Discussion at the NDF AGM focused on Kete project including some statements of real support for Kete and its principles.

A theme of NDF was the acknowledgement of what Kete has achieved but also some concern from users who hear that the software is in a very vulnerable position and requires support to get to a point of sustainability. Joann Ransom raised a matter at the AGM to highlight the risks of the software project.  We want to recap the situation and keep dialogue happening.

Rabid remain committed to the technical upgrade and discussion has included possible options that would bring the technology through to sustainability.  Kete should be a strong candidate for financial commitment, as the current state of the application is unacceptable and installations are vulnerable as they run Ruby on Rails 2 and no longer receive security coverage.

We remain convinced that Kete has potential to grow in a new stage of community users and stakeholders beyond New Zealand.  The uses are diverse, compelling and there is strong value in sourcing, identifying and safeguarding digital records of our community heritage.

Need a refresher? Take a look at the development proposal here.

We have many organisations who might consider the digital community heritage elements of Kete in new sites through New Zealand and throughout Australia.   But new entrants have modern expectations of software and the existing Kete instances themselves suffer from poor usability that make it harder for everyday people to participate where they are installed.

We need to achieve a commercial balance/sustainability and activate the existing user groups to demonstrate that this project is worth supporting and can grow and help more communities.

Wondering what this means?   Here’s an overview of what needs to be put in place

• a governance group mandating improvements to the project.  The fundamentals of this group are being set up during this first phase of redevelopment
• installs of Kete to be participating in a support agreement that will include allowance for improvements to the core project on an ongoing basis
• there may be a model where supported hosting for vanilla Kete installs would make the software more accessible to very small community organisations  (e.g. Pukekura Park) and provide assurance that data is archived, backed up and so on
• a commercial partner with a commitment to the core software project for the medium term

In the longer-term, Kete should be developing out features for better telling stories in our communities, and linking content within items.  Rabid have many ideas about this and hope to see this discussion pull together a roadmap that we can engage in.

Hi there,

It seems timely to write an update on the Kete project.  As you know the project got a green-light in July 2013 and we began in August.   We have a team of two, Rob Ramsay and Breccan McLeod working through this phase and we’ve been putting in steady time on the upgrade.

We work in 2 weekly sprints, and report to JD in Levin, Leith Haarhoff of Palmerston North City Library and Andy Neale at Digital NZ.

We are delighted that Palmerston North City Libraries have contributed $5k towards the project, which reduces risk in the project and it is great to have their involvement as an advocate for the project and a potential future user.

It would be fair to characterise progress through the initial sprints as somewhat slower than we had hoped on the Ruby on Rails upgrade (i.e. completing the work taking longer than estimated which is common in software).  This may mean the work through the Mongo migration and web app aspects also might be at the pessimistic end of our estimates.  One of the benefits of the phased approach is that we are tackling these critical risks right now and we are laying a path for the community to develop new features which are easier to draw in funding for. Coupled with fortnightly meetings, it gives us an ability to keep steady progress and keep the governance group in touch.

Here’s the latest from last week’s meeting:

• The dependent software (gems) have been modified to allow updating the core Rails software.
• The site’s code has been modified to run with the new version of Rails.
• Parts of the software are running (creating/migrating databases), and the server is now able to run.
• Bugs in the software happen when a page is requested, and these are being worked on.
• The next phase is converting the database to Mongo, and should start in a few days.
• After that there will be more bugs evident, which will need to be worked through. We expect this to be a lot of work.
• Some of these bugs are tied to the parts of the system using zebra and the old database, meaning the site won’t be operational till after the mongo change.

It would be great if you dropped a note to comment on the updates, and where your Kete community is at.  Also any suggestions of contributions of stories for the new Kete blog created by JD are welcome.

Josh Forde of Rabid talking about the Cuba St Project, which is running in Wellington as part of the New Zealand Fringe, and how Kete could contribute to similar local history projects.

Josh Forde on the Cuba Street Project and Kete from Rabid Technologies on Vimeo.

Does anyone else out there think there would be any place for a workshop around the idea of getting the most out of your Kete?

A ‘share whats working’ theme as well as:

• Using Storify with Kete
• Using Historypin with Kete
• Creating an animated gif for small slideshows using Photoshop or Gimp
• Embedding twitter feeds
• Embedding Youtube Videos
• Using audacity to clean up your audio contributions
• Good practise (linking content within kete and too own catalogue to create journey, pointing to other reliable resources, separating Library content from public content by using baskets and index pages).
• Others?

If you think there is a demand for this kind of workshop I would happy to run it from the perspective of a happy tinkerer with an ear to learning other’s ideas…(rather than a programmer). A sort of ‘bring a laptop and be prepared to play with you own kete’ couple of hours?

Harley Couper