The AURA (Assessment of Urban–Rural AI) project aims to apply and validate the GeoAI chatbot JackDaw in real-world settings across Central Macedonia, Greece. The goal is to explore how AI-driven tools can support decision-making in sectors such as spatial planning, rural tourism, and business development, by combining diverse datasets (e.g. environmental, demographic, and infrastructure data) into actionable insights .

To achieve this, AURA focuses on three representative use cases:

  1. Smart Residential Decentralisation, supporting decisions on peri-urban living and housing development;
  2. Sustainable Rural Tourism – “Pink Blossom Valley” (Veria), improving planning and resilience of seasonal tourism;
  3. Data-Driven Rural Business Intelligence (Kilkis Region), enhancing access to market and funding insights for rural businesses.

Phase 1: Usability Evaluation

The first phase of the project focuses on developing a concrete understanding of JackDaw’s context of use. More specifically, the aim was to identify potential user groups and extract key usability issues. The expected outcomes of this phase include a clear description of the users, an understanding of their expectations, needs, and challenges, as well as a mapping of the user journeys they follow in order to achieve their goals.

At the beginning of the project, we conducted desk research to identify and describe potential user groups. This process led to the creation of initial user profiles, which were formalised as user personas (see Figure 1), representing the different types of users expected to interact with JackDaw.

Based on these personas, we then designed a Usability Evaluation, with the aim of identifying usability problems and prioritising them according to their importance and impact.

The usability evaluation was conducted with 9 participants (3 per use case), representing the different user groups identified in the earlier stages.

During the sessions, we collected both observational and self-reported data, combining task-based performance metrics with user feedback. In addition, participants completed the System Usability Scale (SUS)—a widely used, standardised questionnaire for assessing perceived usability.

Participants included:

  1. Smart Residential Decentralisation: a candidate resident (F), a real estate agent (M), and a housing developer (F)
  2. Sustainable Rural Tourism (Veria): a local tourism business owner (F), a travel agent (F), and a member of the tourism organisation of Veria (F)
  3. Data-Driven Rural Business Intelligence (Kilkis): a local producer (M), a wholesaler of local products (F), and a cooperative executive (M)

The sessions followed a standard usability testing approach, combining observation, task-based interaction, and qualitative feedback collection, in line with UX best practices.

IMG_20260330_193907

20260330_121405

Key Findings: Usability Challenges

The initial results highlight several important usability and reliability issues that affect the overall user experience. At the same time, participants also identified valuable strengths that point to JackDaw’s potential as a decision-support tool:

  1. Accessibility & Performance
    1. Significant delays when using the tool on slow or unstable networks, raising concerns about accessibility for rural users.
  2. Interaction & Interface Issues
    1. Difficulty editing selecting areas on the map (users must redraw from scratch).
    2. The “select area” functionality is not easily discoverable.
    3. Instructions for selecting areas are not visible when the chat is active.
    4. The text input field changes position on the screen, creating an inconsistency in the User Interface.
    5. Errors may occur after long inactivity when an area is selected.
  3. Language & Input Limitations
    1. High sensitivity to spelling and minor variations (e.g. different grammatical forms or accents in Greek place names).
    2. Difficulty understanding user intent even when queries are correctly formulated.
    3. Requests for alternative keywords instead of interpreting user input.
  4. Content Relevance & Accuracy
    1. Responses sometimes contain:
      1. Incorrect or misleading information,
      2. Overly generic answers not tailored to the selected region,
      3. References to incorrect or irrelevant locations (e.g. Czech Republic instead of Kilkis, or answering for a different region than the one requested),
      4. Mismatched datasets (e.g. airport data for agricultural queries).
    2. In some cases, the tool asks users to provide data it should already have access to.
  5. User Experience & Cognitive Load
    1. Responses are often too long and require equally long prompts to refine them. This increases user effort and disrupts the conversational flow.
    2. The tool sometimes exposes technical/internal elements (e.g. references to “Raven” or requests for confirmation to access SQL databases), which are not meaningful to end users and may create confusions.
  6. Positive Aspects & Emerging Value
    1. Users appreciated the combined use of map and text, which supports spatial understanding and makes results more tangible.
    2. The system’s ability to anticipate follow-up questions helps guide users through their exploration.
    3. JackDaw often provided additional insights and information that users had not initially considered, adding value to the decision-making process.

What These Findings Mean

Overall, the first phase confirms that JackDaw demonstrates clear potential as a GeoAI decision-support tool, particularly in its ability to combine spatial and textual information and enrich user thinking with additional insights.

However, this potential is currently constrained by a set of critical usability and reliability issues that impact accessibility, trust, and ease of use. The findings point to the need for:

  • Improved performance in low-connectivity environments, especially for rural contexts,
  • More robust natural language understanding, including tolerance to spelling and linguistic variation,
  • Increased accuracy and contextual relevance of responses,
  • A more intuitive and stable user interface, and
  • A clearer separation between user-facing functionality and internal system processes.

Addressing these challenges will be essential for reducing user effort, strengthening trust, and enabling smoother, more effective interaction. At the same time, the positive feedback highlights a strong foundation to build upon, particularly in supporting exploratory, data-driven decision-making across diverse rural–urban contexts.

Next Steps

We are currently conducting an in-depth analysis of the usability evaluation results, aiming to further synthesise the insights gathered during Phase 1. This process will allow us to develop detailed user journeys, capturing how different user groups interact with JackDaw, as well as their key pain points, needs, and expectations.

At the same time, these findings are directly informing the design and planning of Phase 2, where we will engage stakeholders through collaborative workshops to explore JackDaw’s broader value, limitations, and potential in real decision-making contexts.

Stay tuned for the next update as we continue to refine and validate GeoAI tools for rural–urban development.